# Project export: Stream - Smart AI Email Agents This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2025 - Tagline: Email DOES suck. It takes forever to go through, especially if you gave up years ago, trying to sort your inbox. Stream makes this easy through Categorization, Email Chat, Auto Reply, and Smart Search - Devpost: https://devpost.com/software/stream-smart-ai-email-agents - GitHub: https://github.com/edwar-d/Stream_ - Video: https://player.vimeo.com/video/1095359753?byline=0&portrait=0&title=0#t=0 - Team: 1 GitHub contributor(s) — Edward Lu (3 commits) ## Devpost submission (written by the team) ### Inspiration As students juggling classes, work, and extracurriculars, email is a crucial tool—but Gmail’s interface often feels clunky and outdated. Important messages get buried, spam gets through, and finding what you need is harder than it should be. At Team Stream, we envisioned a Gmail experience that’s smarter, more intuitive, and powered by AI. So we built Stream, an AI-enhanced email system that simplifies inbox management and introduces conversational interactions, making your email feel more like your assistant, not your chore. ### How we built it We built Stream using Flask as our backend framework and JavaScript/HTML for the frontend interface. Key technologies and components: Google OAuth for secure login and permissioned access to users' Gmail inboxes. Gemini API (via Google’s Vertex AI) to: Categorize emails into smart folders (e.g., Work, School, Promotions, Events). Power an AI email assistant that allows users to talk to their inbox—ask it things like "Show me all unread school-related emails from this week" or "Summarize emails I got today." Gmail API to fetch, label, and manipulate email threads in real-time. Flask routes serve dynamic content and relay user interactions between the frontend and Gemini. ### Challenges we ran into Integrating OAuth and Gmail API was tricky. Dealing with scopes, tokens, and threading models required careful debugging and sandbox testing. ### Accomplishments we're proud of Created a functional AI email assistant with natural language understanding. Successfully classified emails using Gemini with high accuracy, and auto-labeled them into categories. Built a full-stack web app that integrates several complex APIs under one cohesive user experience. Demoed real-time email conversation with the Gemini-powered agent. Smart reply templates powered by Gemini—get quick, tone-matching suggestions for professional, casual, or apologetic emails. Save time and reply right. ### What we learned How to work with Google’s suite of APIs, including Gmail, OAuth, and Gemini. Build production-ready Flask endpoints and serve dynamic content. Fine-tuning prompts and understanding prompt engineering for email-related tasks. Frontend/backend coordination using asynchronous requests and Google’s data models. ### What's next Mobile version so users can manage smart folders and AI chat on the go Google Chrome Extension that overlays directly onto Gmail, allowing users to manage smart folders and chat with the AI assistant within the Gmail interface itself. Further Gemini training to understand tone and urgency ## README (from the GitHub repository) A modern email management system that uses AI to automatically categorize your emails with a beautiful Next.js frontend and Flask backend. ## Architecture - **Flask Backend** (Port 5000): Handles Gmail API, OAuth authentication, and AI categorization - **Next.js Frontend** (Port 3000): Modern React-based UI with server-side rendering ## Prerequisites - Python 3.8+ - Node.js 16+ - Gmail API credentials (credentials.json) ## Setup Instructions ### 1. Flask Backend Setup ```bash # Install Python dependencies pip install -r requirements.txt # Ensure you have credentials.json in the root directory # (Download from Google Cloud Console with Gmail API enabled) ``` ### 2. Next.js Frontend Setup ```bash # Navigate to the Next.js directory cd stream-nextjs # Install Node.js dependencies npm install # Return to root directory cd .. ``` ## Starting the Application ### Method 1: Manual Start (Recommended for Development) **Terminal 1 - Start Flask Backend:** ```bash python app.py ``` This starts the Flask server on http://localhost:5000 **Terminal 2 - Start Next.js Frontend:** ```bash cd stream-nextjs npm run dev ``` This starts the Next.js server on http://localhost:3000 ### Method 2: Quick Test You can test if everything is working by: 1. Start Flask backend: `python app.py` 2. Test health endpoint: Visit http://localhost:5000/api/health 3. Start Next.js frontend: `cd stream-nextjs && npm run dev` 4. Visit http://localhost:3000 ## Features - **Gmail Integration**: Secure OAuth2 authentication with Gmail - **AI Categorization**: Uses Google's Gemini 2.0 Flash for intelligent email categorization - **Real-time Processing**: Live progress tracking during categorization - **Modern UI**: Beautiful, responsive interface built with Next.js and Tailwind CSS - **Email Viewer**: Rich email content display with HTML support, image handling, and link previews - **Custom Categories**: Support for user-defined categorization queries - **Fallback System**: Graceful degradation to rule-based categorization if AI fails ## Usage 1. **Login**: Visit http://localhost:3000 and authenticate with Google 2. **View Inbox**: Browse your emails in a modern interface 3. **Categorize**: Click the magic wand icon or visit /categorize 4. **Custom Queries**: Add specific categorization instructions like "by priority" or "by project" 5. **View Results**: Browse categorized emails in an organized grid layout ## API Endpoints ### Flask Backend (Port 5000) - `GET /` - Authentication check - `GET /login` - Start OAuth flow - `GET /api/emails` - Fetch emails - `GET /api/email/` - Get specific email - `POST /categorize` - Start categorization - `GET /categorize_status/` - Check categorization progress - `GET /categorize_results/` - Get categorization results ### Next.js Frontend (Port 3000) - All routes proxy to Flask backend for API calls - Modern React pages for UI ## Troubleshooting ### Flask Won't Start - Check if Python dependencies are installed: `pip install -r requirements.txt` - Verify credentials.json is in the root directory - Test import: `python -c "import app; print('OK')"` ### Next.js Won't Start - Check if Node.js dependencies are installed: `cd stream-nextjs && npm install` - Verify Node.js version: `node --version` (should be 16+) ### Categorization Fails - Ensure Flask backend is running on port 5000 - Check if Gemini API key is valid in utils.py - System will fallback to rule-based categorization if AI fails ### Authentication Issues - Verify credentials.json has correct redirect URIs - Check that Gmail API is enabled in Google Cloud Console - Clear browser cookies and try again ## Development Notes - The system is designed to be fault-tolerant with multiple fallback mechanisms - AI categorization uses Google's Gemini 2.0 Flash model - Session management handles concurrent categorization requests - The frontend gracefully handles backend failures with mock data ## File Structure ``` Stream/ ├── app.py # Flask backend main file ├── utils.py # Email processing and AI categorization ├── requirements.txt # Python dependencies ├── credentials.json # Google OAuth credentials (not in repo) ├── templates/ # Flask HTML templates (legacy) ├── static/ # Static assets and CSS └── stream-nextjs/ # Next.js frontend ├── src/app/ # Next.js app router pages ├── src/components/ # React components └── src/lib/ # Utility libraries ``` ## License This project is licensed under the MIT License - see the LICENSE file for details. ## Detected evidence (automated analysis) Indexed codebase: 45 recognized source files, 539 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (57 of 57) ``` .gitignore app.py blueprints/__init__.py blueprints/email_functions/emails.py blueprints/folder_view/categories.py credentials.json main_new.py main.py README.md requirements.txt saved_categories/55502f40dc8b7c769880b10874abc9d0_categories.json saved_categories/9617cecc856330ab272f4443d29a1591_categories.json saved_categories/9617cecc856330ab272f4443d29a1591_folders.json static/css/categorize.css static/css/email-viewer.css static/css/inbox.css static/css/loading.css static/js/email-viewer.js static/js/inbox.js static/README_EmailViewer.md static/saved_categories/9617cecc856330ab272f4443d29a1591_categories.json stream-nextjs/.gitignore stream-nextjs/eslint.config.mjs stream-nextjs/next.config.ts stream-nextjs/package.json stream-nextjs/postcss.config.mjs stream-nextjs/src/app/api/categorize-results/[sessionId]/route.ts stream-nextjs/src/app/api/categorize-status/[sessionId]/route.ts stream-nextjs/src/app/api/categorize/route.ts stream-nextjs/src/app/api/email/[id]/mark-read/route.ts stream-nextjs/src/app/api/email/[id]/route.ts stream-nextjs/src/app/api/emails/route.ts stream-nextjs/src/app/api/load-inbox/route.ts stream-nextjs/src/app/api/logout/route.ts stream-nextjs/src/app/categorize/loading/page.tsx stream-nextjs/src/app/categorize/page.tsx stream-nextjs/src/app/categorize/results/[sessionId]/page.tsx stream-nextjs/src/app/globals.css stream-nextjs/src/app/inbox/page.tsx stream-nextjs/src/app/layout.tsx stream-nextjs/src/app/loading/page.tsx stream-nextjs/src/app/login/google/authorized/page.tsx stream-nextjs/src/app/login/page.tsx stream-nextjs/src/app/logout/page.tsx stream-nextjs/src/app/page.tsx stream-nextjs/src/components/EmailPopup.tsx stream-nextjs/src/components/EmailViewer.tsx stream-nextjs/src/lib/api.ts stream-nextjs/tsconfig.json templates/categorize_loading.html templates/categorize.html templates/chat.html templates/inbox.html templates/loading.html templates/login.html templates/test.html utils.py ``` ### Dependencies - requirements.txt: Flask@==2.3.3, Flask-CORS@==4.0.0, google-api-python-client@==2.103.0, google-auth@==2.23.3, google-auth-oauthlib@==1.0.0, google-generativeai@==0.3.2 - stream-nextjs/package.json: @eslint/eslintrc@^3, @tailwindcss/postcss@^4, @types/node@^20, @types/react@^19, @types/react-dom@^19, axios@^1.10.0, eslint@^9, eslint-config-next@15.3.4, next@15.3.4, react@^19.0.0, react-dom@^19.0.0, tailwindcss@^4, typescript@^5 ### Recent commits (newest first) - Update README.md - update - project_final - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### static/README_EmailViewer.md ```markdown # EmailViewer Implementation for Static Frontend This implementation brings the rich email viewing capabilities from the Next.js EmailViewer component to the static HTML frontend, allowing you to view emails with HTML rendering, Markdown support, link previews, and image handling. ## Features ### ✨ Rich Content Display - **HTML Rendering**: Safely renders HTML emails with security sanitization - **Markdown Support**: Automatically detects and renders Markdown content - **Plain Text**: Fallback for plain text emails with proper formatting ### 🔒 Security Features - **HTML Sanitization**: Removes dangerous scripts and elements - **Image Blocking**: Images are blocked by default for security - **Safe Links**: External links open in new tabs with security attributes ### 🎨 Interactive Elements - **Link Previews**: Automatic preview cards for popular sites (GitHub, YouTube, LinkedIn, Twitter) - **Image Toggle**: Show/hide images with a single click - **Responsive Design**: Works on desktop and mobile devices ### 📎 Attachment Support - **File Display**: Shows attached files with icons and sizes - **Download Ready**: Prepared for file download functionality ## Files Structure ``` static/ ├── js/ │ ├── email-viewer.js # Main EmailViewer class implementation │ └── inbox.js # Updated to integrate with EmailViewer ├── css/ │ ├── email-viewer.css # Styles for the EmailViewer component │ ├── inbox.css # Existing inbox styles │ └── ... └── README_EmailViewer.md # This file templates/ ├── inbox_enhanced.html # Enhanced inbox template with EmailViewer └── inbox.html # Original inbox template ``` ## Usage ### Access the Enhanced Inbox 1. **Start the Flask Backend**: ```bash python app.py ``` 2. **Visit the Enhanced Inbox**: ``` http://localhost:5000/inbox/enhanced ``` ### Integration with Existing Code The EmailViewer automatically integrates with the existing inbox functionality: 1. **Email Selection**: Click on any email in the list to view it 2. **Rich Content**: HTML and Markdown content is automatically detected and rendered 3. **Image Control**: Use the "Show Images" button to toggle image display 4. **Link Previews**: URLs in emails automatically generate preview cards ### API Integration The EmailViewer connects to the Flask backend using these endpoints: - `GET /api/email/{id}` - Fetch email content with HTML and plain text - `POST /api/email/{id}/mark-read` - Mark email as read - `DELETE /api/email/{id}` - Delete email (with confirmation) ## Key Components ### EmailViewer Class (`email-viewer.js`) ```javascript class EmailViewer { // Main methods: selectEmail(emailId) // Load and display an email renderEmail() // Render email with rich content sanitizeHTML(html) // Security sanitization renderMarkdown(text) // Markdown rendering extractAndPreviewLinks(content) // Generate link previews to [truncated — 2517 more characters] ``` ### requirements.txt ``` # Web Framework Flask==2.3.3 Flask-CORS==4.0.0 # Google OAuth and Gmail API google-auth-oauthlib==1.0.0 google-api-python-client==2.103.0 google-auth==2.23.3 # Google Generative AI (Gemini) google-generativeai==0.3.2 # Standard library dependencies (included for completeness, but usually pre-installed) # These are typically part of Python standard library but listing for clarity: # - base64 (built-in) # - datetime (built-in) # - re (built-in) # - html (built-in) # - os (built-in) # - threading (built-in) # - time (built-in) # - uuid (built-in) # - pickle (built-in) # - json (built-in) # - random (built-in) # - asyncio (built-in) # - concurrent.futures (built-in) ``` ### stream-nextjs/package.json ``` { "name": "stream-nextjs", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "axios": "^1.10.0", "next": "15.3.4", "react": "^19.0.0", "react-dom": "^19.0.0" }, "devDependencies": { "@eslint/eslintrc": "^3", "@tailwindcss/postcss": "^4", "@types/node": "^20", "@types/react": "^19", "@types/react-dom": "^19", "eslint": "^9", "eslint-config-next": "15.3.4", "tailwindcss": "^4", "typescript": "^5" } } ``` ### main.py ```python from flask import Flask, request, redirect, session, url_for, render_template, jsonify, g from flask_cors import CORS from google_auth_oauthlib.flow import Flow from googleapiclient.discovery import build from google.auth.transport.requests import Request import os import threading import time import uuid from flask import jsonify import pickle import json from datetime import timedelta import base64 from utils import EmailClient, gen_categories, QuerySaver, CategoryStorage import random import asyncio from concurrent.futures import ThreadPoolExecutor import ssl from googleapiclient.errors import HttpError import httplib2 # Configure httplib2 for better SSL handling httplib2.Http.force_exception_to_status_code = True # Create a thread pool executor executor = ThreadPoolExecutor(max_workers=4) app = Flask(__name__) CORS(app, origins=["http://localhost:5000"], supports_credentials=True) # Enable CORS for static frontend app.secret_key = 'your-secret-key-here-make-this-consistent' # Use consistent secret key app.permanent_session_lifetime = timedelta(minutes=15) # Configure session cookie settings for cross-origin requests app.config.update( SESSION_COOKIE_SAMESITE='Lax', # Changed from 'None' to 'Lax' for better compatibility SESSION_COOKIE_SECURE=False, # Set to True in production with HTTPS SESSION_COOKIE_HTTPONLY=False, # Allow JavaScript access for debugging SESSION_COOKIE_DOMAIN=None, # Allow cookies across localhost ports SESSION_COOKIE_PATH='/' ) # Add custom JSON filter for templates @app.template_filter('tojson') def to_json(value): return json.dumps(value) # OAuth 2.0 configuration - using your exact redirect URIs CLIENT_SECRETS_FILE = 'credentials.json' SCOPES = ['https://www.googleapis.com/auth/gmail.readonly', 'https://www.googleapis.com/auth/gmail.modify'] REDIRECT_URI = 'http://localhost:5000/login/google/authorized' # Matches your credentials.json # Initialize email client email_client = EmailClient() query = QuerySaver() # Initialize the OAuth flow def get_flow(): return Flow.from_client_secrets_file( CLIENT_SECRETS_FILE, scopes=SCOPES, redirect_uri=REDIRECT_URI ) # Note: We can't use custom HTTP clients with credentials in Google API client # The credentials object already contains the HTTP transport configuration @app.before_request def before_request(): """Set up shared objects for each request""" g.email_client = email_client g.query = query @app.route('/') def index(): """Main entry point - serve static frontend""" if 'credentials' not in session: return redirect(url_for('login_page')) return redirect(url_for('loading_page')) @app.route('/inbox') def inbox(): """Main inbox route after loading""" if 'credentials' not in session: return redirect(url_for('index')) try: if email_client.has_service and email_client.email_list!=None: email_list = email_client.email_list profile = email_client.profile user_email = profile.get('emailAddress', 'Unknown') if profile else 'Unknown' else: creds = pickle.loads(session['credentials']) service = build('gmail', 'v1', credentials=creds) email_client.add_service(service) email_list = email_client.get_messages(max_results=100) profile = email_client.profile user_email = profile.get('emailAddress', 'Unknown') if profile else 'Unknown' return render_template('inbox.html', emails=email_list, user_email=user_email, total_emails=len(email_list)) except Exception as e: return f"Error: {str(e)} Try again" @app.route('/api/health') def health_check(): """Health check endpoint""" return jsonify({ 'status': 'healthy', 'message': 'Flask backend is running', 'authenticated': 'credentials' in session, 'session_keys': list(session.keys()) }) @app.route('/api/debug') def debug_session(): """Debug session information""" return jsonify({ 'session_keys': list(session.keys()), 'has_credentials': 'credentials' in session, 'session_id': request.cookies.get('session'), 'cookies': dict(request.cookies) }) @app.route('/api/session-test') def session_test(): """Test endpoint to verify session functionality""" if 'test_counter' not in session: session['test_counter'] = 0 session['test_counter'] += 1 return jsonify({ 'session_id': request.cookies.get('session', 'No session cookie'), 'test_counter': session['test_counter'], 'session_keys': list(session.keys()), 'has_credentials': 'credentials' in session, 'has_oauth_state': 'oauth_state' in session, 'oauth_state': session.get('oauth_state', 'No state stored') }) @app.route('/login') def login_page(): """Serve the login page""" return render_template('login.html') @app.route('/loading') def loading_page(): """Serve the loading page""" if 'credentials' not in session: return redirect(url_for('login_page')) return render_template('loading.html') @app.route('/categorize/loading') def categorize_loading_page(): """Serve the categorize loading page""" if 'credentials' not in session: return redirect(url_for('login_page')) # Get parameters from URL session_id = request.args.get('session_id', '') user_email = request.args.get('user_email', '') total_emails = request.args.get('total_emails', '0') return render_template('categorize_loading.html', session_id=session_id, user_email=user_email, total_emails=total_emails) @app.route('/chat') def chat_page(): """Dedicated chat interface for interacting with ema [truncated — 5312 more characters] ``` ### stream-nextjs/src/app/layout.tsx ```typescript import type { Metadata } from "next"; import "./globals.css"; export const metadata: Metadata = { title: "Stream - AI Email Organization", description: "Organize less, flow more. AI that anticipates, so you don't have to organize", }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return ( {children} ); } ``` ### stream-nextjs/src/app/page.tsx ```typescript 'use client'; import Link from "next/link"; import { useEffect, useState } from 'react'; export default function HomePage() { const [isLoading, setIsLoading] = useState(true); useEffect(() => { // Check authentication status with Flask backend const checkAuth = async () => { try { console.log('[DEBUG] Main page - Checking authentication with Flask backend'); const response = await fetch('http://localhost:5000/', { method: 'GET', credentials: 'include', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' } }); console.log('[DEBUG] Main page - Auth check response status:', response.status); if (response.ok) { const data = await response.json(); console.log('[DEBUG] Main page - Auth check response:', data); if (data.authenticated) { console.log('[DEBUG] Main page - User is authenticated, redirecting to loading'); // User is authenticated, redirect to loading page window.location.href = '/loading'; return; } } console.log('[DEBUG] Main page - User is not authenticated, showing login page'); // User is not authenticated, show the login page setIsLoading(false); } catch (error) { console.error('[DEBUG] Main page - Error checking authentication:', error); // If Flask backend is not available, show login page setIsLoading(false); } }; checkAuth(); }, []); if (isLoading) { return (

Checking authentication...

); } return (

Stream

Organize less, flow more. AI that anticipates, so you don't have to organize

Sign in with Google

Features

Clean, familiar user interface
Search That Reads Your Mind
Smart Categories, Zero Effort
Responsive design for all devices
); } ``` ### stream-nextjs/src/app/logout/page.tsx ```typescript 'use client'; import { useEffect } from 'react'; export default function LogoutPage() { useEffect(() => { // Clear any stored authentication data if (typeof window !== 'undefined') { localStorage.clear(); sessionStorage.clear(); // In a real app, you would also clear cookies and revoke tokens // For demo purposes, just redirect to home setTimeout(() => { window.location.href = '/'; }, 1000); } }, []); return (

Stream

Signing you out...

You will be redirected to the login page shortly.

); } ``` ### stream-nextjs/src/app/login/page.tsx ```typescript 'use client'; import { useEffect } from 'react'; export default function LoginPage() { useEffect(() => { // Redirect to Flask backend for Google OAuth const handleLogin = () => { window.location.href = 'http://localhost:5000/login'; }; // Auto-redirect after a short delay to show the loading state const timer = setTimeout(handleLogin, 1500); return () => clearTimeout(timer); }, []); const handleManualLogin = () => { window.location.href = 'http://localhost:5000/login'; }; return (

Stream

Organize less, flow more. AI that anticipates, so you don't have to organize

Features

Clean, familiar user interface
Search That Reads Your Mind
Smart Categories, Zero Effort
Responsive design for all devices
); } ``` ### stream-nextjs/src/app/loading/page.tsx ```typescript 'use client'; import { useEffect, useState } from 'react'; import { useRouter } from 'next/navigation'; export default function LoadingPage() { const router = useRouter(); const [messageIndex, setMessageIndex] = useState(0); const loadingMessages = [ "Loading your experience...", "Connecting to your Gmail", "Fetching your emails", "Almost ready" ]; useEffect(() => { const messageInterval = setInterval(() => { setMessageIndex((prev) => (prev + 1) % loadingMessages.length); }, 2000); const loadInboxData = async () => { try { console.log('[DEBUG] Loading page - Making direct request to Flask backend'); // Make direct request to Flask backend with credentials const response = await fetch('http://localhost:5000/api/load-inbox', { method: 'GET', credentials: 'include', // Include cookies for cross-origin requests headers: { 'Accept': 'application/json', 'Content-Type': 'application/json' } }); console.log('[DEBUG] Loading page - Flask response status:', response.status); if (!response.ok) { const errorData = await response.json(); console.log('[DEBUG] Loading page - Flask error:', errorData); if (errorData.redirect) { router.push(errorData.redirect); return; } throw new Error(errorData.error || `HTTP ${response.status}`); } const data = await response.json(); console.log('[DEBUG] Loading page - Success, received data for:', data.user_email); // Store data in sessionStorage for the inbox page sessionStorage.setItem('inboxData', JSON.stringify(data)); // Small delay for better UX setTimeout(() => { router.push('/inbox'); }, 1000); } catch (error) { console.error('[DEBUG] Loading page - Error loading inbox:', error); // Check if it's an authentication error if (error instanceof Error && error.message.includes('Not authenticated')) { console.log('[DEBUG] Loading page - Authentication error, redirecting to login'); router.push('/'); return; } // Retry after 3 seconds for other errors setTimeout(loadInboxData, 3000); } }; // Start loading process setTimeout(loadInboxData, 1500); return () => { clearInterval(messageInterval); }; }, [router]); return (
{/* Floating Particles */}
{[...Array(9)].map((_, i) => (
))}

Stream

{loadingMessages[messageIndex]} ...