# Project export: TreeSync

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: TreeSync simplifies your life by transforming your emails into Google Calendar events seamlessly. Never worry about missing important events again – let our AI take care of it for you!
- Devpost: https://devpost.com/software/treesync
- GitHub: https://github.com/daniellee0115/TreeSync
- Video: https://www.youtube.com/embed/-OPAtqxOeiM?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — daniellee0115 (8 commits)

## Devpost submission (written by the team)

### Inspiration

TreeSync was inspired by the need to streamline the organization and synchronization of events from emails directly into a user’s calendar. In our fast-paced world, managing and transferring important dates and times from emails to a calendar can be tedious. We aimed to automate this process, saving time and reducing the risk of missing critical appointments or deadlines.

### What it does

TreeSync automatically scans a user's incoming emails for event-related information, determines the importance of each event, color-codes it accordingly, and inserts the event into the user’s Google Calendar. It acts as a personal assistant, ensuring that all essential events from emails are scheduled without manual input.

### How we built it

We built TreeSync using Python, leveraging the Gmail API to access and parse emails and the Google Calendar API to manage calendar events. For natural language processing, we integrated Together.ai's LLM (specifically Llama-2-70b-chat), which was instrumental in interpreting the contents of emails and evaluating the significance of each potential event. The powerful AI capabilities provided by Together.ai allowed us to implement sophisticated event detection and classification, enabling our application to understand the nuances of human language in various email formats.

### Challenges we ran into

One of the main challenges we faced with TreeSync was implementing a system that could continuously scan emails in the background without user intervention. We used ngrok and Google Cloud's Pub/Sub and mailbox watch to connect Gmail's push notification to ngrok's public website, then using flask and ngrok, we connected the push notification to a local server, which allowed us to call a function to add an event to the calendar every time Google Cloud pushed a notification. Additionally, crafting an algorithm that could reliably extract dates, times, and event details from the diverse formatting found in emails was non-trivial. To aid in this, we integrated Together.ai's LLM for natural language understanding, which improved our success rate. Another significant hurdle was handling edge cases such as ambiguous date formats, events happening in a set amount of time as opposed to on a specific date, events scheduled only in the subject, etc. Ensuring that TreeSync could make intelligent decisions in these scenarios was a substantial part of our development efforts. Finally, we had to figure out how to make this whole experience seamless and easy for new users. We therefore created an application using pyinstaller, enabling new users to just launch the application once and then just forget about it, letting the AI do the work in the background.

### Accomplishments we're proud of

We managed to create a system that works effectively in the background, requiring minimal user setup and providing a high level of automation for email event synchronization. We worked effectively as a team, got closer in the process, and learned a lot!

### What we learned

Throughout this project, we've learned about the intricacies of working with APIs, the challenges of natural language processing, and the importance of creating a user-friendly automated system. We've also gained experience in handling JSON data structures and learned more about best practices in Python coding and backend engineering. We also learned how to work effectively in a team given the short timeframe.

### What's next

Moving forward, we aim to refine TreeSync’s event-detection algorithms for even greater accuracy. We plan to implement a user interface to allow for individual preference settings and manual override options. Additionally, we will explore expanding TreeSync’s capabilities to other email and calendar platforms, further broadening its usability.

## README (from the GitHub repository)

# TreeSync - Sync Your Calendar With Your Emails Using LLMs!

pyinstaller --add-data='./credentials.json:.' --noconsole --icon=logo.png --name=TreeSync treesync.py


## Detected evidence (automated analysis)

Indexed codebase: 3 recognized source files, 13 KB.
- Python (language) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (5 of 5)

```
.DS_Store
credentials.json
llm.py
README.md
treesync.py
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- Update README.md
- Fixed errors regarding late response.
- Delete TreeSync.app directory
- Update llm.py
- Update credentials.json
- Deleted token.json
- Fixed minor bugs related to returning None as the answer.
- Added remote fetch functionality and dealt with edge cases. Added a fully functional app.
- Deleted unnecessary files.
- Added basic functionality code.
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### llm.py

```python
import time

def llmCall(message):
    # message is an array with 0th value being system instructions and 1st value being user input.
    import requests
    import json
    url = "https://api.together.xyz/v1/chat/completions"

    payload = {
        "model": "meta-llama/Llama-2-70b-chat-hf",
        "max_tokens": 512,
        "stop": ["</s>", "[/INST]"],
        "temperature": 0.1,
        "top_p": 0.1,
        "top_k": 50,
        "repetition_penalty": 1,
        "n": 1,
        "messages": [
            {
                "role": "system",
                "content": message[0]
            },
            {
                "role": "user",
                "content": message[1]
            }
        ]
    }
    headers = {
        "accept": "application/json",
        "content-type": "application/json",
        "Authorization": "Bearer <key>"
    }
    time.sleep(0.5)
    response = requests.post(url, json=payload, headers=headers)
    if "choices" in json.loads(response.text):
        return json.loads(response.text)["choices"][0]["message"]["content"]
    return None

```

### treesync.py

```python
import os
import json
import base64
import time
import subprocess
from llm import llmCall
from google.oauth2.credentials import Credentials
from google.auth.transport.requests import Request
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from flask import Flask, request

base_dir = os.path.dirname(__file__)
prev_startHistoryId = None
prev_email_ids = {}
prev_events = {}
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', 'https://www.googleapis.com/auth/calendar']

def get_gmail_service():
    """Authenticate and create a Gmail service."""
    # Load credentials from a JSON file or perform OAuth2 authentication
    creds = None
    if os.path.exists(os.path.join(base_dir, 'token.json')):
        creds = Credentials.from_authorized_user_file(os.path.join(base_dir,'token.json'))
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                os.path.join(base_dir, 'credentials.json'), SCOPES)
            creds = flow.run_local_server(port=0)
        # Save credentials for future use
        with open(os.path.join(base_dir, 'token.json'), 'w') as token:
            token.write(creds.to_json())

    # Create Gmail service
    service = build('gmail', 'v1', credentials=creds)
    return service

def get_calendar_service():
    """Authenticate and create a Google Calendar service."""
    creds = None
    if os.path.exists(os.path.join(base_dir, 'token.json')):
        creds = Credentials.from_authorized_user_file(os.path.join(base_dir,'token.json'))
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                os.path.join(base_dir, 'credentials.json'), SCOPES)
            creds = flow.run_local_server(port=0)
        with open(os.path.join(base_dir, 'token.json'), 'w') as token:
            token.write(creds.to_json())

    service = build('calendar', 'v3', credentials=creds)
    return service

def get_latest_email(service, startHistoryId):
    """Fetch the latest email from Gmail."""
    # Get the list of messages
    results = service.users().messages().list(userId='me', labelIds=['INBOX']).execute()
    messages = results.get('messages', [])

    if not messages:
        print("No messages found.")
        return []
    new_email_ids = set()
    if startHistoryId is None:
        # Get the ID of the latest message
        new_email_ids.add(messages[0]['id'])
    else:
        # Gets email ids of all emails after push notification
        response = service.users().history().list(userId='me', startHistoryId=startHistoryId).execute()
        changes = response.get('history', [])
        new_email_ids = set()
        for change in changes:
            for message_change in change.get('messages', []):
                message_id = message_change['id']
                if message_id not in prev_email_ids:
                    prev_email_ids[message_id] = 1
                    new_email_ids.add(message_id)

    # Retrieve the latest messages
    new_email_ids = list(new_email_ids)
    prev_startHistoryId = new_email_ids[0]
    messages = [service.users().messages().get(userId='me', id=message_id).execute() for message_id in new_email_ids]
    return messages

def return_email_content(message):
    """Print the contents of the email."""
    # Date of email
    date = llmCall(["You are an agent that receives a string with a date and time, along with other information. Your job is to return the date (year, month, day, name of day (Ex. Thursday, Friday, Saturday, etc.)) and time (hour and minutes in a 24-hour military time system, where you convert AM/PM to the correct hour count). Do not output any filler words or extra words or extra spaces or indents that is not the answer. Make sure to put the answer in one line. Make sure that the answer has no unnecessary space in the front or in the back. Ex: Friday, January 1, 2023 18:00:00", message['payload']['headers'][1]['value']])
    payload = message['payload']
    parts = payload.get('parts', [payload])
    for part in parts:
        if part['mimeType'] == 'text/plain' or part['mimeType'] == 'text/html':
            data = part['body']['data']
            time.sleep(1)
            # Body of email
            body = llmCall(["You are an agent that receives an html code which contains a text body. Your job is to return the text body only. Do not return anything else.", base64.urlsafe_b64decode(data).decode('utf-8')[:3000]])
            break
    headers = payload.get('headers', [])
    for header in headers:
        if header['name'].lower() == 'subject':
            subject = header['value']
            break
    return date, subject, body

def determine_event_importance(email_subject, email_body):
        importance_prompt = f"""

        You are an agent that receives an email event with a subject and body. Your task is to assess the importance level of the event based on the provided information.

        Consider the following factors when assessing importance:
        - Capitalization: Words in all caps or with excessive capitalization may indicate urgency.
        - Specific keywords: Look for words like 'urgent', 'important', 'deadline', 'action required', etc.
        - Tone: Evaluate the overall tone of the email. Does it convey a sense of urgency or importance?
        - Timing: Is there a specific deadline or time-sensitive information mentioned?

        Based on these considerations, classify the event as one of the following:
        1. 'Very important': The email contains critical information that requires immediate attention or action. Or if the event is taking place very soon from when the email was received.
        2. 'Important': The email is significant 
[truncated — 5767 more characters]
```