# Project export: EduDeals

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: UC Berkeley AI Hackathon 2026
- Tagline: EduDeals is a student-discount finder web application that helps students browse, search, and save discounts — with a personalized feed that surfaces deals for your school based on your .edu email.
- Devpost: https://devpost.com/software/edudeals
- GitHub: https://github.com/oh-a-cai/edudeals
- Demo: https://edu-deals.vercel.app/
- Team: 1 GitHub contributor(s) — Oliver Cai (11 commits)

## Devpost submission (written by the team)

### Inspiration

As current college students, we felt that many students were not taking advantage of the discounts provided by their .edu email. The problem isn't the lack of discounts, its the lack of accessibility. Discounts are scattered across brand sites, campus pages, directories, and random GitHub lists, with no single place to look. Most of the deals you find are expired or not for your school. We wanted one clean feed that knows you're a student the moment you sign in with your .edu email, and shows only the deals that are relevant to you.

### What it does

EduDeals is a searchable, filterable catalog of verified student discounts. Sign in with your .edu email to unlock a paginated grid of deals you can search by brand or description, filter by category, and sort by name, expiry, or highest % off. A "For your school" tab matches deals to your campus from your email domain, and a "Saved" tab keeps your favorites. Expired deals are hidden automatically, expiring-soon deals are flagged, and your filters live in the URL so a view is shareable. The catalog itself is kept fresh by a Python pipeline that scrapes and extracts new deals into the database on its own.

### How we built it

The frontend is React 19 + TypeScript + Tailwind CSS v4, talking directly to Supabase for Postgres storage and email/password auth — no backend of our own. Discounts are populated by a standalone Python pipeline: httpx + BeautifulSoup fetch campus and city deal pages, Google Gemini extracts structured deals from messy HTML in one batched call, clean GitHub markdown lists are parsed deterministically with regex, and everything is upserted into Postgres via psycopg using ON CONFLICT (redemption_url) so re-runs update existing rows instead of duplicating.

### Challenges we ran into

Messy, inconsistent school data. The school field comes in as display names ("UC San Diego"), bare domains ("chapman.edu"), and the keyword "all" — matching a user's email domain to the right deals needed a normalization heuristic rather than an exact lookup. Deduping without losing real deals. The same brand legitimately appears on multiple campuses; we key dedup on normalized brand + host so the same listing scraped twice on one site collapses, but the same brand across different campuses doesn't. LLM cost and reliability. Sending every page in full to Gemini was wasteful, so we route only messy HTML to the LLM and parse clean, structured markdown deterministically instead.

### Accomplishments we're proud of

A self-updating catalog built with the scraper means the deal list isn't hand-maintained, which is paired with personalized per-school feeds, shareable URL state, persistent dark mode, and a uniform card grid that adapts cleanly no matter how long a deal's description is. And a data pipeline that mixes LLM extraction with deterministic parsing to keep both speed and accuracy.

### What we learned

How to combine an LLM with deterministic parsing so each approach does what it's best at, how to design dedup/upsert logic that's idempotent across re-runs, and how much of building a real product is taming inconsistent data rather than writing features. On the frontend, we got a lot more comfortable with React state that needs to stay in sync with the URL and localStorage at the same time.

### What's next

A proper domain → school mapping (or a user-set school) so display-name campuses match correctly. Scheduling the scraper (cron/serverless) so the catalog refreshes automatically. One-click copy for promo codes, deal ratings/verification by students, and expiry notifications. Broadening sources beyond the initial California campuses.

## README (from the GitHub repository)

# EduDeals

A student-discount finder web application built with React, TypeScript, and Supabase. EduDeals helps students browse, search, and save verified discounts — with a personalized feed that surfaces deals for your school based on your `.edu` email. Discounts are populated automatically by a Python scraping pipeline.

---

## Overview

Students sign in with a `.edu` email and land on a searchable grid of discounts. From there, they can:

- **Browse all deals** — paginated grid of discount cards, 9 per page
- **Search & filter** — match by brand/description, filter by one or more categories, and sort by name, expiry, or highest % off
- **View your university-specific deals** — a dedicated tab matches discounts against the user's email domain (e.g. `@g.ucla.edu` → schools containing "ucla")
- **Save favorites** — heart any deal to add it to a personal "Saved" tab, persisted per user
- **Read full details** — long descriptions are clamped on the card, with a "Read more" link that opens a detail modal when text actually overflows
- **Toggle dark mode** — toggle persisted to `localStorage`, defaulting to OS preference, with a glow-on-hover card effect

Filtered/sorted views are written to the URL, so a search can be shared or survives a page reload. The last-viewed tab is also remembered across sessions.

Behind the scenes, a standalone Python pipeline discovers deals from campus pages and seed lists, extracts them with Gemini, and upserts them into the same Postgres database the frontend reads from.

---

## Tech Stack

- **Frontend:** React 19, TypeScript, Tailwind CSS v4 (`@tailwindcss/vite`)
- **Backend:** Supabase (Postgres database + email/password auth) — the React app talks to Supabase directly
- **Data pipeline:** Python, `httpx` + `BeautifulSoup` for fetching/parsing, **Google Gemini** for deal extraction, `psycopg` for writing straight to Postgres
- **Build tooling:** Vite, ESLint

The scraper writes to the same database out of band — the frontend never runs it directly.

---

## Project Structure

```
client/
└── src/
    ├── components/   # DiscountGrid, DiscountCard, AuthBar, ThemeToggle, Toast, ResetPassword
    ├── library/      # Supabase client + useSession / useSavedDiscounts hooks
    ├── scraper/      # Python pipeline (main.py, config.py, database.py, pipeline_*.py, tests)
    ├── types.ts      # the Discount type
    └── App.tsx       # app shell
```

---

## How to Run

### Prerequisites

- [Node.js](https://nodejs.org/) (v18+ recommended)
- npm
- Python 3.10+ (for the scraper)
- A [Supabase](https://supabase.com) project

### Frontend

**1. Clone the repository**
```bash
git clone https://github.com/yourusername/edudeals.git
```

**2. Navigate to the client directory**
```bash
cd edudeals/client
```

**3. Install dependencies**
```bash
npm install
```

**4. Configure environment variables**

Create a `.env` (or `.env.local`) in `client/` with your Supabase project credentials:
```
VITE_SUPABASE_URL=https://<your-project>.supabase.co
VITE_SUPABASE_ANON_KEY=<your-anon-key>
```

**5. Set up the database**

Supabase needs two tables:

| Table | Columns |
|---|---|
| `discounts` | `id`, `brand`, `description`, `discount_percent` (text, e.g. `"50% off"`), `category`, `redemption_url` (unique — the scraper upserts on it), `expires_at` (nullable), `created_at`, `school` (nullable — a school name, an email domain like `chapman.edu`, or the literal `all`), `tags` (text array) |
| `saved_discounts` | `user_id`, `discount_id` (a user's hearted discounts) |

Enable **Email/Password** auth in Supabase for sign-in, favoriting, and the school feed.

**6. Start the dev server**
```bash
npm run dev
```
> Runs at `http://localhost:5173`.

---

### Other Commands

```bash
npm run build     # type-check + production build to dist/
npm run preview   # serve the production build locally
npm run lint      # run ESLint
```

---

## Data Pipeline (Python Scraper)

`src/scraper/` is a standalone pipeline that discovers student-discount listings, extracts structured deals, and upserts them into the `discounts` table. The frontend never runs it — it's a separate batch job, run on demand or on a schedule.

`main.py` orchestrates four stages:

1. **Collect pages** — fetches a curated list of campus/city deal pages (`LOCAL_URLS` in `config.py`), raw GitHub markdown deal lists (`GITHUB_SEED_URLS`), and search-discovered directory pages. `httpx` handles fetching, with **ScraperAPI** as a 403 fallback.
2. **Extract deals** — messy campus HTML is cleaned and sent in a single batched call to **Google Gemini**, which returns structured deals; clean GitHub markdown lists are parsed deterministically with regex (no LLM quota used).
3. **Dedupe & merge** — collapses the same merchant repeated on one site (by normalized brand + host) while keeping genuinely distinct deals; GitHub wins on cross-source brand collisions.
4. **Upsert** — writes to `public.discounts` via a `psycopg` async connection pool, using `ON CONFLICT (redemption_url)` so re-runs update existing rows instead of duplicating. A `scraped_output_debug.json` dump is written alongside for inspection.

### Running the Scraper

**1. Navigate to the scraper directory**
```bash
cd src/scraper
```

**2. Install dependencies**
```bash
pip install -r requirements.txt
```

**3. Run the pipeline**
```bash
python main.py
```
> Reads `client/.env` (two levels up).

**Required and optional environment variables:**
```
DATABASE_URL=postgresql://...      # required — Supabase → Project Settings → Database → Connection string (URI), NOT the REST URL
GEMINI_API_KEY=...                 # required for LLM extraction of campus HTML
SERPAPI_KEY=...                    # optional — brand/directory discovery
SCRAPERAPI_KEY=...                 # optional — fallback for pages that return 403
```

To add a source, add its URL to `LOCAL_URLS` or `GITHUB_SEED_URLS` in `config.py` — the semantic extractor generalizes across layouts, so no per-site CSS selectors are needed. The `test_*.py` files cover DB connectivity, inserts, and discovery in isolation.

---

## Deployment (Vercel)

The app lives in the `client/` directory, not the repo root. In your Vercel project settings, set **Root Directory** to `client` and add the two `VITE_SUPABASE_*` environment variables. Vite is auto-detected — no further build configuration needed.

---

## License

MIT

## Detected evidence (automated analysis)

Indexed codebase: 26 recognized source files, 96 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Supabase (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Vercel (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (34 of 34)

```
client/.gitignore
client/eslint.config.js
client/index.html
client/package.json
client/src/App.tsx
client/src/components/AuthBar.tsx
client/src/components/DiscountCard.tsx
client/src/components/DiscountGrid.tsx
client/src/components/ResetPassword.tsx
client/src/components/SortDropdown.tsx
client/src/components/ThemeToggle.tsx
client/src/components/Toast.tsx
client/src/index.css
client/src/library/supabase.ts
client/src/library/useSavedDiscounts.ts
client/src/library/useSession.ts
client/src/main.tsx
client/src/scraper/config.py
client/src/scraper/database.py
client/src/scraper/main.py
client/src/scraper/pipeline_global.py
client/src/scraper/pipeline_local.py
client/src/scraper/requirements.txt
client/src/scraper/scraped_output_debug.json
client/src/scraper/test_database.py
client/src/scraper/test_db_insert.py
client/src/scraper/test_discovery.py
client/src/types.ts
client/tsconfig.app.json
client/tsconfig.json
client/tsconfig.node.json
client/vite.config.ts
package.json
README.md
```

### Dependencies

- client/package.json: @eslint/js@^10.0.1, @supabase/supabase-js@^2.108.2, @tailwindcss/vite@^4.3.1, @types/node@^24.12.3, @types/react@^19.2.14, @types/react-dom@^19.2.3, @vitejs/plugin-react@^6.0.1, eslint@^10.3.0, eslint-plugin-react-hooks@^7.1.1, eslint-plugin-react-refresh@^0.5.2, globals@^17.6.0, react@^19.2.6, react-dom@^19.2.6, tailwindcss@^4.3.1, typescript@~6.0.2, typescript-eslint@^8.59.2, vite@^8.0.12
- client/src/scraper/requirements.txt: beautifulsoup4, google-genai, httpx, psycopg[binary,pool], pydantic, python-dotenv
- package.json: @tailwindcss/vite@^4.3.1, tailwindcss@^4.3.1

### Recent commits (newest first)

- added confirmation popup for links, created custom dropdown menu for filters
- updated readme to include web scraper details
- included readme file describing the whole project
- updated various frontend designs
- refined school tag
- Merge branch 'main' of https://github.com/oh-a-cai/edudeals
- added scraper
- refined tags and categories as well as added a school tag
- fixed global link discovery
- added local and global link discovery
- first iteration of the local link webscraper
- added favicon and logo, renamed app to EduDeals, set default card sorting to be alphabetical
- added extra sorting options, shareable url filters, and a password reset feature
- implemented multi-select filter and toast notifications
- included a dark mode toggle
- added search bar with filtering, automatic pagenation, and favoriting discounts
- implemented basic user auth and displayed discounts in a grid
- setup supabase connection
- setup boilerplate frontend code
- read me

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "dependencies": {
    "@tailwindcss/vite": "^4.3.1",
    "tailwindcss": "^4.3.1"
  }
}

```

### client/package.json

```
{
  "name": "react",
  "private": true,
  "version": "0.0.0",
  "type": "module",
  "scripts": {
    "dev": "vite",
    "build": "tsc -b && vite build",
    "lint": "eslint .",
    "preview": "vite preview"
  },
  "dependencies": {
    "@supabase/supabase-js": "^2.108.2",
    "react": "^19.2.6",
    "react-dom": "^19.2.6"
  },
  "devDependencies": {
    "@eslint/js": "^10.0.1",
    "@tailwindcss/vite": "^4.3.1",
    "@types/node": "^24.12.3",
    "@types/react": "^19.2.14",
    "@types/react-dom": "^19.2.3",
    "@vitejs/plugin-react": "^6.0.1",
    "eslint": "^10.3.0",
    "eslint-plugin-react-hooks": "^7.1.1",
    "eslint-plugin-react-refresh": "^0.5.2",
    "globals": "^17.6.0",
    "tailwindcss": "^4.3.1",
    "typescript": "~6.0.2",
    "typescript-eslint": "^8.59.2",
    "vite": "^8.0.12"
  }
}

```

### client/src/scraper/requirements.txt

```
psycopg[binary,pool]
httpx
beautifulsoup4
python-dotenv
google-genai
pydantic

```

### client/src/main.tsx

```typescript
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import './index.css'
import App from './App.tsx'

createRoot(document.getElementById('root')!).render(
  <StrictMode>
    <App />
  </StrictMode>,
)

```

### client/src/App.tsx

```typescript
import AuthBar from './components/AuthBar'
import DiscountGrid from './components/DiscountGrid'
import ThemeToggle from './components/ThemeToggle'
import { ToastContainer } from './components/Toast'
import ResetPassword from './components/ResetPassword'
import logo from './assets/logo.png'

export default function App() {
  return (
    <div className="min-h-screen bg-gray-50 dark:bg-gray-950">
      <header className="border-b border-gray-200 bg-white dark:border-gray-800 dark:bg-gray-900">
        <div className="mx-auto flex max-w-5xl items-center justify-between px-6 py-4">
          <div className="flex items-center gap-3">
            <img src={logo} alt="Edudeals Logo" className="w-8" />
            <h1 className="text-xl font-bold text-gray-900 dark:text-gray-100">EduDeals</h1>
          </div>
          <div className="flex items-center gap-3">
            <AuthBar />
            <ThemeToggle />
          </div>
        </div>
      </header>

      <main className="mx-auto max-w-5xl px-6 py-8">
        <DiscountGrid />
      </main>

      <ToastContainer />
      <ResetPassword />
    </div>
  )
}

```

### client/src/scraper/main.py

```python
"""Orchestrates the scraper: curated + discovered URLs -> Gemini -> public.discounts.

Curated LOCAL_URLS and discovery-scout URLs are scraped, batched into ONE Gemini
call, then upserted into the single flat public.discounts table — no scope split.
scraped_output_debug.json keeps an on-disk copy for debugging.
"""
import asyncio
import json
import os
import re
from dataclasses import asdict

from urllib.parse import urlsplit

import httpx

from config import LOCAL_URLS, GITHUB_SEED_URLS
from pipeline_global import discover_local_directories, filter_directories
from pipeline_local import fetch, page_text, parse_deals_batch, parse_github_markdown
from database import Deal, save_deals

# Anchored to this file (not cwd) so the dump always lands in the scraper folder.
_DEBUG_PATH = os.path.join(os.path.dirname(__file__), "scraped_output_debug.json")

# Intent-driven seeds: find regional / campus student-deal directories, not
# individual brand offers.
SEED_QUERIES = [
    "site:.edu 'student discounts' OR 'campus perks' California",
    "downtown association 'student deals' OR 'local discounts'",
]


# Brand-name noise words stripped before comparison, so "Spotify" and "Spotify
# Premium Student Discount" collapse to the same key.
_BRAND_NOISE = re.compile(
    r"\b(premium|student|students|discount|discounts|offer|offers|deal|deals|for|the|pro|plan)\b")


def _norm_brand(name: str) -> str:
    """Brand name -> comparison key: noise words dropped, then alnum-only."""
    b = re.sub(r"[^a-z0-9]+", "", _BRAND_NOISE.sub("", name.lower()))
    return b or re.sub(r"[^a-z0-9]+", "", name.lower())  # all-noise -> full name


def _dedupe_key(d: Deal) -> tuple[str, str] | None:
    """(normalized brand, host). Collapses the same merchant listed twice on the
    same site (e.g. a product in both GitHub files) while KEEPING the same brand on
    different sites — Woodstock's in two cities, Spotify campus vs spotify.com — so
    we never delete a genuinely distinct deal. Null URL -> no key (passes through)."""
    if not d.redemption_url:
        return None
    return (_norm_brand(d.brand), urlsplit(d.redemption_url).netloc.lower())


def dedupe(deals: list[Deal]) -> list[Deal]:
    """Collapse rows for the same merchant on the same site, keeping the first seen.
    Null-URL rows pass through — they're dropped later at the save step anyway."""
    seen: set[tuple[str, str]] = set()
    out: list[Deal] = []
    for d in deals:
        key = _dedupe_key(d)
        if key and key in seen:
            continue
        if key:
            seen.add(key)
        out.append(d)
    return out


def merge_sources(campus: list[Deal], github: list[Deal]) -> list[Deal]:
    """Dedupe within each source by (brand, host); across sources GitHub wins — if a
    brand exists in the GitHub lists, drop the campus rows for that same brand (the
    curated enterprise listing is preferred over a campus-specific one)."""
    github = dedupe(github)
    gh_brands = {_norm_brand(d.brand) for d in github}
    campus = [d for d in dedupe(campus) if _norm_brand(d.brand) not in gh_brands]
    return campus + github


def dump_debug(deals: list[Deal]) -> None:
    """Write the standardized payload to disk for a human eyeball before it hits
    the DB — runs first, so we keep a record even if the upsert fails."""
    with open(_DEBUG_PATH, "w", encoding="utf-8") as f:
        json.dump([asdict(d) for d in deals], f, indent=2, ensure_ascii=False)
    print(f"[main] Wrote {len(deals)} deals -> {_DEBUG_PATH}")


async def collect_global_urls() -> list[str]:
    """Run the discovery scout over all seed queries, filter, dedupe."""
    found: list[str] = []
    for query in SEED_QUERIES:
        found.extend(await discover_local_directories(query))

    urls = filter_directories(found)
    print(f"[main] discovery -> {len(urls)} candidate directories")
    return urls


async def collect_pages(urls: list[str], client: httpx.AsyncClient) -> list[dict]:
    """Fetch (native -> ScraperAPI) each URL and flatten to text, building the
    batch aggregator payload: [{"url": url, "html_text": text}, ...]."""
    pages: list[dict] = []
    for url in urls:
        html = await fetch(url, client)
        if not html:
            print(f"[main] {url} -> unreachable, skipping")
            continue
        pages.append({"url": url, "html_text": page_text(html)})
    return pages


async def main():
    async with httpx.AsyncClient(timeout=30, follow_redirects=True) as client:
        local_pages = await collect_pages(LOCAL_URLS, client)
        # Raw markdown lists: native fetch, same {url, html_text} shape. page_text's
        # HTML cleaner is a harmless passthrough on tag-free markdown.
        github_pages = await collect_pages(GITHUB_SEED_URLS, client)
        global_urls = await collect_global_urls()
        global_pages = await collect_pages(global_urls, client)

    # Campus HTML is messy -> Gemini. The curated GitHub lists are clean structured
    # markdown -> a deterministic regex parser (100% recall, no quota, no LLM call).
    campus_deals = await parse_deals_batch(local_pages + global_pages)
    github_deals = parse_github_markdown(github_pages)
    # Within-source dedupe + GitHub-wins on cross-source brand collisions.
    # (Non-USD region-locked deals were already dropped at parse time.)
    all_deals = merge_sources(campus_deals, github_deals)

    dump_debug(all_deals)

    written = await save_deals(all_deals)
    print(f"[main] discounts upserted: {written} | debug json: {len(all_deals)} "
          f"(campus {len(campus_deals)} + github {len(github_deals)}, deduped)")


if __name__ == "__main__":
    asyncio.run(main())

```

### client/vite.config.ts

```typescript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import tailwindcss from '@tailwindcss/vite'

// https://vite.dev/config/
export default defineConfig({
  plugins: [react(), tailwindcss()],
})

```

### client/index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/png" href="/favicon.png" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>EduDeals</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
</html>

```

### client/eslint.config.js

```javascript
import js from '@eslint/js'
import globals from 'globals'
import reactHooks from 'eslint-plugin-react-hooks'
import reactRefresh from 'eslint-plugin-react-refresh'
import tseslint from 'typescript-eslint'
import { defineConfig, globalIgnores } from 'eslint/config'

export default defineConfig([
  globalIgnores(['dist']),
  {
    files: ['**/*.{ts,tsx}'],
    extends: [
      js.configs.recommended,
      tseslint.configs.recommended,
      reactHooks.configs.flat.recommended,
      reactRefresh.configs.vite,
    ],
    languageOptions: {
      globals: globals.browser,
    },
  },
])

```

### client/src/types.ts

```typescript
export interface Discount {
  id: string
  brand: string
  description: string
  discount_percent: string
  category: string
  redemption_url: string
  expires_at: string | null
  created_at: string
  school: string | null
}

```

[18 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]