# Project export: Nova AI

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Your Automated Personal Assistant.
- Devpost: https://devpost.com/software/nova-your-automated-pa
- GitHub: https://github.com/jp-singaraju/nova-ai
- Result: winner (Interaction Company: Build AI Agents that Proactively Handle Your Email!)
- Team: 1 GitHub contributor(s) — Jathin Pranav Singaraju (8 commits)

## Devpost submission (written by the team)

### Inspiration

The inspiration for Nova came from the overwhelming volume of emails and tasks that professionals face daily. We aimed to create a solution that simplifies task management and reduces cognitive load, allowing users to focus on what truly matters.

### What it does

Nova is an automated email assistant that intelligently processes incoming emails, identifies actionable items, and seamlessly adds them to your calendar. It also sends timely text reminders, ensuring you stay organized and on top of your commitments without the hassle of manual tracking.

### How we built it

We built Nova using natural language processing algorithms to analyze email content and extract relevant tasks. By integrating with calendar APIs and SMS services, we created a smooth workflow that automates task management and communication, making it easy for users to manage their schedules.

### Challenges we ran into

One of the main challenges was accurately interpreting the context of emails to distinguish between urgent tasks and general information. Additionally, ensuring seamless integration with various calendar platforms and messaging services required extensive testing and refinement.

### Accomplishments we're proud of

We are proud of developing a fully functional prototype of Nova that effectively reduces users' daily load by automating task management. Initial user feedback has been overwhelmingly positive, highlighting the assistant's ability to streamline workflows and enhance productivity.

### What we learned

Throughout the development process, we learned the importance of user feedback in refining our algorithms and improving the overall user experience. We also gained insights into the complexities of integrating multiple services to create a cohesive solution.

### What's next

for Nova Moving forward, we plan to enhance Nova's capabilities by incorporating machine learning to improve task recognition and prioritization. Our goal is to expand its features and ultimately launch it as a comprehensive productivity tool that transforms how users manage their daily tasks.

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

Indexed codebase: 5 recognized source files, 14 KB.
- Flask (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code

## Codebase structure (from repository index)

### Files (11 of 11)

```
.DS_Store
.gitignore
authentication.py
credentials.json
gcal.py
gmail.py
main.py
requirements.txt
text_message.py
token_gcal.pickle
token_gmail.pickle
```

### Dependencies

- requirements.txt: aiohappyeyeballs@==2.4.3, aiohttp@==3.10.10, aiohttp-retry@==2.8.3, aiosignal@==1.3.1, annotated-types@==0.7.0, anyio@==4.6.2.post1, attrs@==24.2.0, blinker@==1.8.2, cachetools@==5.5.0, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, distro@==1.9.0, Flask@==3.0.3, frozenlist@==1.4.1, google-api-core@==2.21.0, google-api-python-client@==2.149.0, google-auth@==2.35.0, google-auth-httplib2@==0.2.0, google-auth-oauthlib@==1.2.1, googleapis-common-protos@==1.65.0, h11@==0.14.0, httpcore@==1.0.6, httplib2@==0.22.0, httpx@==0.27.2, icalendar@==6.0.1, idna@==3.10, itsdangerous@==2.2.0, Jinja2@==3.1.4, jiter@==0.6.1, MarkupSafe@==3.0.2, multidict@==6.1.0, oauthlib@==3.2.2, openai@==1.52.0, propcache@==0.2.0, proto-plus@==1.24.0, protobuf@==5.28.2, pyasn1@==0.6.1, pyasn1_modules@==0.4.1, pydantic@==2.9.2, pydantic_core@==2.23.4, PyJWT@==2.9.0, pyparsing@==3.2.0, python-dateutil@==2.9.0.post0, python-dotenv@==1.0.1, requests@==2.32.3, requests-oauthlib@==2.0.0, rsa@==4.9, sinch@==1.0.0, six@==1.16.0, sniffio@==1.3.1, tqdm@==4.66.5, twilio@==9.3.4, typing_extensions@==4.12.2, tzdata@==2024.2, uritemplate@==4.1.1, urllib3@==2.2.3, Werkzeug@==3.0.4, yarl@==1.15.5

### Recent commits (newest first)

- finished main
- Merge pull request #1 from jp-singaraju/gmail-integration
- added text functionality
- updated downloads
- added calendar functionality
- added gmail integration
- added gpt functionality
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
aiohappyeyeballs==2.4.3
aiohttp==3.10.10
aiohttp-retry==2.8.3
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.6.2.post1
attrs==24.2.0
blinker==1.8.2
cachetools==5.5.0
certifi==2024.8.30
charset-normalizer==3.4.0
click==8.1.7
distro==1.9.0
Flask==3.0.3
frozenlist==1.4.1
google-api-core==2.21.0
google-api-python-client==2.149.0
google-auth==2.35.0
google-auth-httplib2==0.2.0
google-auth-oauthlib==1.2.1
googleapis-common-protos==1.65.0
h11==0.14.0
httpcore==1.0.6
httplib2==0.22.0
httpx==0.27.2
icalendar==6.0.1
idna==3.10
itsdangerous==2.2.0
Jinja2==3.1.4
jiter==0.6.1
MarkupSafe==3.0.2
multidict==6.1.0
oauthlib==3.2.2
openai==1.52.0
propcache==0.2.0
proto-plus==1.24.0
protobuf==5.28.2
pyasn1==0.6.1
pyasn1_modules==0.4.1
pydantic==2.9.2
pydantic_core==2.23.4
PyJWT==2.9.0
pyparsing==3.2.0
python-dateutil==2.9.0.post0
python-dotenv==1.0.1
requests==2.32.3
requests-oauthlib==2.0.0
rsa==4.9
sinch==1.0.0
six==1.16.0
sniffio==1.3.1
tqdm==4.66.5
twilio==9.3.4
typing_extensions==4.12.2
tzdata==2024.2
uritemplate==4.1.1
urllib3==2.2.3
Werkzeug==3.0.4
yarl==1.15.5

```

### main.py

```python
# import functions
import os
from gmail import get_latest_message, extract_info, download_attachments
from gcal import create_calendar_event
from text_message import send_text
from authentication import authenticate
from flask import Flask

app = Flask(__name__)

def get_message_and_add_downloads():
    # write the main function
    store_dir = "./attachments/"
    os.makedirs(store_dir, exist_ok=True)

    # authenticate to google api
    service = authenticate("gmail")
    latest_email = get_latest_message(service, "me")
    extracted_info = extract_info(latest_email)

    # if there are attachments, download them
    if latest_email.get("attachments"):
        msg_id = latest_email["id"]
        download_attachments(service, "me", msg_id, store_dir)
    return extracted_info

def add_event_to_calendar(message):
    # add an event to calendar
    service = authenticate("calendar")
    event = create_calendar_event(service, message)
    return event

# run the app route for the webhook
@app.route('/webhook')
def process_incoming_email():
    message = get_message_and_add_downloads()
    add_event_to_calendar(message)
    send_text(message)
    return

if __name__ == "__main__":
    app.run(port=5000)
```

### authentication.py

```python
# import functions
import os
import pickle
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request

# request all access (permission to read/send/receive emails, manage the inbox, and more)
GMAIL_SCOPES = ["https://mail.google.com/"]
CALENDAR_SCOPES = ['https://www.googleapis.com/auth/calendar']

def authenticate(type):
    # pickle stores the user's access and refresh tokens, and is created automatically when the authorization flow completes for the first time
    creds = None
    if (type == "gmail"):
        file = "token_gmail.pickle"
        version = "v1"
    else:
        file = "token_gcal.pickle"
        version = "v3"
    
    # check pickle file
    if os.path.exists(file):
        with open(file, "rb") as token:
            creds = pickle.load(token)
    
    # if no credentials available, let the user log in
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            if (type == "gmail"):
                flow = InstalledAppFlow.from_client_secrets_file('credentials.json', GMAIL_SCOPES)
            else:
                flow = InstalledAppFlow.from_client_secrets_file('credentials.json', CALENDAR_SCOPES)
            creds = flow.run_local_server(port=0)

        # save credentials for next run
        with open(file, "wb") as token:
            pickle.dump(creds, token)
    
    # return the service
    return build(type, version, credentials=creds)
```

### text_message.py

```python
# import functions
import os
from dotenv import load_dotenv
import openai
from sinch import SinchClient
import json

# load env vars
load_dotenv()
openai.api_key = os.getenv("OPENAI_API_KEY")

def parse_message(message):
    # parse input message to get information
    example_conversation = {"phone_number": "+19725039779", "message": "Hey John, Nova here. Jeff just told me about you. Are you interested in driving the minivan around the city to ship our new merch?"}
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=[
                {"role": "system", "content": f"You are an assistant that analyzes conversations between two people about a third person. Your task is to extract a valid phone number in the format +XXXXXXXXXXX (e.g., +12145067234) from the conversation, analyze the discussion to understand key points about the third person, and synthesize this information into a concise message directed to the third person (the phone number owner). If both a valid phone number and relevant information are present, return a JSON object with phone_number and message keys. The message should be written as if coming from the recipient of the original conversation, addressing the third person directly about the discussed topic. Include the recipient's name if mentioned. Make sure to return null if either the phone number or relevant information is missing. Ignore conversation parts unrelated to the third person or the extracted number. For example: {example_conversation}."},
                {"role": "user", "content": f"Extract the phone number and message content from this text: {message}"}
            ],
        response_format={"type": "json_object"}
        )
    return response.choices[0].message.content

def send_text(message):
    # init the sinch client
    sinch_client = SinchClient(
        key_id="c282428c-f911-4d89-abce-5fe7c638e7ba",
        key_secret="33.sSkemtozGFojdCfbykvfYZJ",
        project_id="5244202a-7235-4684-9fb1-1b96dc48a89b"
    )

    # parse the message
    parsed_data = parse_message(message)
    print(parsed_data)
    if parsed_data.lower() == "null":
        print("NO TEXT MESSAGE SENT.")
        return

    # convert the string response to a dictionary
    data = json.loads(parsed_data)

    # send the SMS
    send_batch_response = sinch_client.sms.batches.send(
        body=data["message"],
        to=[data["phone_number"]],
        from_="+12064743901",
        delivery_report="none"
    )
    print("TEXT MESSAGE SENT.")
    return send_batch_response
```

### gmail.py

```python
# import functions
import openai
import json
from dotenv import load_dotenv
import os
import base64
import email

# load env vars
load_dotenv()
openai.api_key = os.getenv("OPENAI_API_KEY")

def get_latest_message(service, user_id):
    # get the latest message
    messages = service.users().messages().list(userId=user_id, maxResults=1, q="is:inbox").execute()
    if 'messages' in messages:
        latest_msg_id = messages['messages'][0]['id']
        return get_message_info(service, user_id, latest_msg_id)
    else:
        print("No messages found.")
        return None

def get_message_info(service, user_id, msg_id):
    # extract email details
    message = service.users().messages().get(userId=user_id, id=msg_id, format='raw').execute()
    msg_str = base64.urlsafe_b64decode(message['raw'].encode("utf-8")).decode("utf-8")
    fin_msg = email.message_from_string(msg_str)
    email_data = {
        "id": msg_id,
        "sender": fin_msg["from"],
        "recipients": fin_msg["to"],
        "subject": fin_msg["subject"],
        "date": fin_msg["date"],
        "plain_text": extract_plain_text(fin_msg),
        "attachments": extract_attachments(fin_msg)
    }
    return email_data

def extract_plain_text(fin_msg):
    # get the plain text from message
    if fin_msg.is_multipart():
        for part in fin_msg.walk():
            if part.get_content_type() == "text/plain":
                return part.get_payload(decode=True).decode("utf-8")
    else:
        return fin_msg.get_payload(decode=True).decode("utf-8")
    return ""

def extract_attachments(fin_msg):
    # find the attachments in this message
    attachments = []
    if fin_msg.is_multipart():
        for part in fin_msg.walk():
            if part.get_content_type() not in ["text/plain", "text/html"] and part.get_filename():
                attachment = {
                    "filename": part.get_filename(),
                    "fileType": part.get_content_type(),
                }
                attachments.append(attachment)
    return attachments

def download_attachments(service, user_id, msg_id, store_dir):
    # call it on the message
    message = service.users().messages().get(userId=user_id, id=msg_id).execute()
    parts = message['payload'].get('parts', [message['payload']])

    # process each part
    def process_part(part):
        if part.get('filename') and 'body' in part:
            if part['body'].get('attachmentId'):
                attachment = service.users().messages().attachments().get(userId=user_id, messageId=msg_id, id=part['body']['attachmentId']).execute()
                file_data = base64.urlsafe_b64decode(attachment['data'].encode('UTF-8'))
            elif part['body'].get('data'):
                file_data = base64.urlsafe_b64decode(part['body']['data'].encode('UTF-8'))
            else:
                return
            path = os.path.join(store_dir, part['filename'])
            with open(path, 'wb') as f:
                f.write(file_data)
    
    # iterate through all parts
    for part in parts:
        process_part(part)
        for subpart in part.get('parts', []):
            process_part(subpart)
    print(f"DOWNLOADS RECEIVED.")

def extract_info(data):
    # extract info using gpt model
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that extracts important information from email dictionary data. You should extract the important information from the given email (represented as a dictionary) and return only the relevant fields such as id, sender, recipients, subject, date, plain_text, and attachments as a JSON string. Only return the JSON string."},
            {"role": "user", "content": f"Here is the given email in dictionary format: {data}"}
        ],
    )
    extracted_info = response.choices[0].message.content
    print("MESSAGE RETRIEVED.")
    return json.loads(extracted_info)
```

### gcal.py

```python
# import functions
import json
from datetime import datetime, timedelta
import openai
import os
from dotenv import load_dotenv

# load env vars
load_dotenv()
openai.api_key = os.getenv("OPENAI_API_KEY")

def determine_message_type(message):
    # figure out if this is an event or reminder or other
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=[
            {"role": "system", "content": "Determine if the given message is an event or a reminder. Output a JSON object with a single key 'type' and value either 'event' or 'reminder'."},
            {"role": "user", "content": f"Determine message type: {message}"}
        ],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)["type"]

def process_event(message):
    # process the information if it is an event
    event_categories = {"summary", "location", "description", "start", "end", "attendees"}
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=[
            {"role": "system", "content": f"You are a helpful assistant tasked with extracting information from a message and organizing it into the given categories: {event_categories}. Your goal is to create a structured calendar event based on the message content. For each category, extract relevant details from the message, and if no information is provided for a specific category, set its value to null. When information is incomplete or implicit, infer values logically based on the context, such as estimating location, description, or participants. For the attendees category, include the recipient and sender email in the format 'attendees'; only include these two emails: [{{'email': 'recipient_email', 'email': 'sender_email'}}]. If no time zone is specified, use the default 'timeZone': 'America/Los_Angeles', and format the start and end times like 'start': {{'dateTime': '2015-05-28T09:00:00-07:00', 'timeZone': 'America/Los_Angeles'}}, 'end': {{'dateTime': '2015-05-28T17:00:00-07:00', 'timeZone': 'America/Los_Angeles'}}. Only create an event if a start time is explicitly stated or strongly implied in the message. If a start time is present, estimate an appropriate end time by inferring the event’s duration based on the type of event or message content. The final output should be a JSON object, where each category is a key and the corresponding value is either the extracted or inferred information. Focus on extracting key details, especially the event’s start time, end time, title, location, attendees, and time zone, while ensuring the event is scheduled only when relevant information is provided."},
            {"role": "user", "content": f"Here is the given message: {message}"}
        ],
        response_format={"type": "json_object"}
    )
    return json.loads(response.choices[0].message.content)

def process_reminder(message):
    # process the information if it is an reminder
    response = openai.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=[
            {"role": "system", "content": "Extract reminder information from the given message. Output a JSON object with keys: summary, start. For start, use format: {'dateTime': 'YYYY-MM-DDTHH:MM:SS-07:00', 'timeZone': 'America/Los_Angeles'}. If no specific time is mentioned, schedule for the next appropriate time."},
            {"role": "user", "content": f"Extract reminder information: {message}"}
        ],
        response_format={"type": "json_object"}
    )
    reminder_data = json.loads(response.choices[0].message.content)
    start_time = datetime.fromisoformat(reminder_data["start"]["dateTime"].replace("-07:00", ""))
    end_time = start_time + timedelta(minutes=30)
    reminder_data["end"] = {
        "dateTime": end_time.isoformat() + "-07:00",
        "timeZone": "America/Los_Angeles"
    }
    return reminder_data

def create_calendar_event(service, message):
    # create the calendar event based on the given information
    try:
        message_type = determine_message_type(message)
        if message_type == "event":
            data = process_event(message)
        else:
            data = process_reminder(message)
        calendar_item = service.events().insert(calendarId="primary", body=data, sendUpdates="all").execute()
        print("CALENDAR UPDATED.")
        return json.dumps({"status": "Calendar event created", "event": calendar_item})
    except Exception as e:
        print("ERROR CREATING CALENDAR EVENT: " + str(e))
        return
```