# Project export: AI Personal Finance Assistant

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: OpenAI Build Week
- Tagline: See the patterns behind your spending
- Devpost: https://devpost.com/software/ai-personal-finance-assistant
- GitHub: https://github.com/ol1ak5/AI-Personal-Finance-Assistant
- Video: https://www.youtube.com/embed/_VLKwRPPVik?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — Olya (11 commits)

## Devpost submission (written by the team)

### Inspiration

In a world of constant online shopping, subscriptions, and one-click purchases, it's easier than ever to spend money without really noticing where it goes. Most people have dozens of recurring payments and hundreds of transactions every month, yet a bank statement is still just a long list of numbers. It tells you what you spent, but not why, or what patterns are hiding underneath. So, I wanted to build something that could change that. My goal was to build a project that is useful and easy to use, yet still powerful "under the hood". That's why I decided to unite AI and machine learning to create an app that gives people a clear understanding of their spending without requiring them to be finance experts or spend hours analyzing spreadsheets. It's a project I wanted to be meaningful from day one, something I'd genuinely use myself, and technically solid enough to be more than a weekend distraction.

### What it does

AI Personal Finance Assistant analyzes a bank statement and uncovers the spending habits hidden behind the transactions. Users upload a bank export (.csv or .xlsx, up to 5 MB), and the app automatically processes it, groups merchants into meaningful spending patterns using K-means clustering, and identifies recurring spending behaviors. Then, GPT-5.6 gives each spending pattern a meaningful name and generates a concise explanation in plain English. Instead of simply saying that spending increased by 20% compared to last month, the app explains why it increased – whether it was due to subscriptions, recurring purchases, one-off expenses, or changes in spending habits. Users can compare their spending over the last month, three months, six months, or across their entire transaction history to understand how their finances evolve over time. How I built it The application is built entirely in Python, with Codex acting as a development partner throughout the project. Uploaded files are parsed into pandas DataFrames, merchant names are normalized, and a set of features is extracted for each merchant, including purchase frequency, average spending, timing regularity, and price consistency. Those features are clustered using scikit-learn's K-means algorithm to identify meaningful spending patterns. To protect user privacy, GPT-5.6 never receives raw transaction data. Instead, it only sees aggregated statistics for each cluster, which it uses to generate human-readable pattern names and personalized spending insights through the OpenAI API. Everything is presented through a responsive Streamlit dashboard with interactive Plotly visualizations, making the results easy to explore on desktop, tablet, and mobile devices. Challenges I ran into While building this project, I faced several challenges, but two stood out the most: Working around Streamlit's styling limitations. Streamlit is a great option for quickly building and deploying applications, especially for beginners, but customizing its appearance can be surprisingly difficult. The built-in charts are quite opinionated, and even something as simple as creating a donut chart with rounded segments required hours of experimenting, tweaking, and manually patching the underlying chart configuration. Making the AI provide real insights instead of obvious observations. Getting GPT to describe a chart is easy, yet it doesn't add much value. The real challenge was prompting it to identify genuinely useful patterns instead of repeating what users could already see. After a lot of experimentation, I focused the summaries on five key areas: the biggest spending change, subscriptions, recurring habits, unusual one-off expenses, and anything that deserves a closer look. Accomplishments that I'm proud of Building my first frontend application from scratch and turning it into a polished, fully functional product. Taking the project from a rough idea to a deployed application within the hackathon timeframe. Creating a clean, modern interface that focuses on insights rather than overwhelming users with numbers. Building an AI summary that explains why spending changed instead of simply describing the charts. Delivering a responsive dashboard that works well across desktop, tablet, and mobile devices. What I learned This was my first time building a frontend application from scratch and my first experience working with Codex as a true development partner rather than just a coding assistant. Along the way, I learned how to design and deploy an interactive Streamlit dashboard, apply K-means clustering to noisy real-world financial data, engineer meaningful features from transaction histories, and integrate GPT-5.6 to generate useful, human-friendly insights. Most importantly, I learned what it takes to turn an idea into a polished, working product in just a few days.

### What's next

Downloadable PDF reports that users can save or share. Support for additional languages beyond English. A light theme alongside the current dark interface. Lightweight budgeting, allowing users to set spending targets for categories or patterns and receive alerts when they exceed them. AI-powered recommendations for unnecessary subscriptions and recurring expenses, with suggestions on where users could save money. Long-term spending forecasts and trend predictions for users with enough historical data.

## README (from the GitHub repository)

# 💰 AI Personal Finance Assistant
### *See the patterns behind your spending* · *Apps for Your Life*

---

## 🚀 See it in action

Open the [live app](https://your-ai-personal-finance-assistant.streamlit.app) to explore your spending patterns instantly with an interactive dashboard that turns raw transactions into clear, actionable insights. 

## 🎯 The Problem

Bank statements hide more than they show. They tell you every transaction, but not the story behind your spending. Recurring subscriptions, everyday purchases, and hidden spending patterns are buried in hundreds of line items, making it difficult to understand where your money actually goes.

## 💡 The Solution

AI Personal Finance Assistant transforms raw transaction data into clear spending insights. By combining machine learning with GPT-5.6, it discovers meaningful spending patterns and explains them in plain English, helping you to understand your spending in just a few seconds.

## ✨ Core Features

| Feature | Description |
|---|---|
| **Reveal the habits behind the transactions** | The app identifies up to five spending patterns using frequency, typical amount, timing, and consistency, not merchant names alone |
| **Turn data into a clear story** | GPT-5.6 gives each pattern a meaningful label and writes a plain-English summary |
| **See change over time** | Compare the last month, three months, six months, or your full history without redefining the underlying patterns |
| **Use the app anywhere** | Explore your spending patterns comfortably on a computer, tablet, or mobile phone |
| **Keep your data private** | Files are processed in memory and never stored. AI receives only aggregated pattern evidence. Never raw transaction history |

## 🚀 Quick Start

### Local Development

```bash
# Install dependencies
uv sync --group dev

# Run offline (with mock analysis)
MOCK_LLM=true uv run streamlit run app.py

# Run with GPT-5.6 (requires OPENAI_API_KEY)
OPENAI_API_KEY=sk-... uv run streamlit run app.py
```

Demo data is bundled (`data/spending_demo.csv`). No upload needed to explore.

### Testing

```bash
# Run all 51 tests offline (no OpenAI calls)
MOCK_LLM=true uv run pytest

# Run a specific test
MOCK_LLM=true uv run pytest tests/test_llm.py::test_summary_with_question_rejected -v
```

### Deployed Instance

Visit **[your-ai-personal-finance-assistant.streamlit.app](https://your-ai-personal-finance-assistant.streamlit.app)**. The app redeploys automatically when changes are pushed to `main`. Its `OPENAI_API_KEY` is securely configured in Streamlit Cloud Secrets.

## 🧠 How Codex Contributed to The Final Result

- **Product development** - Codex turned the initial idea into a finished personal-finance product.
- **Engineering** - Codex helped to write and improve the code behind the Streamlit app, and detect the bugs.
- **Accuracy & consistency** - Codex tracked changes across code and documentation, keeping updates synchronized, and reducing the risk of inconsistencies as the project evolved.
- **App design** - Codex supported continuous design iteration across the dashboard layout, charts, responsive desktop/mobile experience.
- **Demo data** - Codex helped to create a realistic demo dataset to showcase features without using sensitive data.
- **Project delivery** - Codex helped to prepare the README, licence, repository structure, etc.

## 🏆 How GPT-5.6 Shaped the Product

- **Pattern labels** - GPT-5.6 turns the raw evidence into meaningful labels based on spending frequency, typical amount, and consistency.
- **Spending summary** - GPT-5.6 translates the analysis into five practical insights: the biggest change, subscriptions, recurring habits, one-off expenses, and items worth a closer look.
- **Clear language** - GPT-5.6 presents insights in natural, conversational language instead of technical terminology.
- **Currency-aware results** - GPT-5.6 uses the currency detected from the uploaded statement so amounts are presented in the user’s original currency.
- **Natural merchant names** - GPT-5.6 makes merchant references human-friendly instead of repeating raw bank-statement descriptions.

## 🎥 Demo Video

📺 **[Watch the Demo on YouTube](https://youtu.be/_VLKwRPPVik)**

The demo covers:
- The hidden-spending problem and why it matters
- How Codex was used in the development workflow
- Why pairing statistical clustering with GPT-5.6 is the right approach
- Live walkthrough with synthetic demo data (patterns, spending habits, AI summary)

## 🧩 Built With

**OpenAI GPT-5.6** · **Python 3.11** · **Streamlit** · **Pandas** · **scikit-learn** · **Plotly** · **pytest**

## 📄 License

Copyright © 2026 Olga Aksenova.

The code in this repository is licensed under the **[Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)** – see [LICENSE](LICENSE) for the full text.

*Built for OpenAI Build Week · Apps for Your Life*


## Detected evidence (automated analysis)

Indexed codebase: 19 recognized source files, 206 KB.
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- Streamlit (technology) — detected in the code

## Codebase structure (from repository index)

### Files (26 of 26)

```
.gitignore
.python-version
.streamlit/config.toml
app.py
core/__init__.py
core/clustering.py
core/features.py
core/llm.py
core/merchants.py
core/parser.py
data/spending_demo.csv
docs/DESIGN.md
docs/merchant-habits-design.md
docs/merchant-habits-plan.md
docs/PLAN.md
LICENSE
pyproject.toml
README.md
tests/__init__.py
tests/conftest.py
tests/test_clustering.py
tests/test_features.py
tests/test_llm.py
tests/test_merchants.py
tests/test_parser.py
uv.lock
```

### Dependencies

- pyproject.toml: fpdf2@>=2.7, kaleido@>=0.2.1, openai@>=1.40, openpyxl@>=3.1, pandas@>=2.2, plotly@>=5.22, scikit-learn@>=1.5, streamlit@>=1.36

### Recent commits (newest first)

- Fix grammar in the problem description
- Revert eyebrow text to match demo video
- Add missing AI to the app eyebrow text
- Fix grammar in the problem description
- docs: fix deployed instance link text mismatch
- docs: update deployed instance link text
- Merge remote README grammar fixes
- docs: fix grammar and add demo video link
- Fix grammar and improve clarity in README
- docs: clarify Codex and GPT contributions
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### docs/merchant-habits-design.md

```markdown
# Merchant habits section: design

Date: 2026-07-19. Status: agreed in conversation, pending final review.
Visual mockup (real demo-data numbers): claude.ai artifact "Merchant habits mockup".

## Goal

The dashboard groups merchants into spending patterns but never shows the
per-merchant detail behind them. A new "Merchant habits" section lists every
merchant, organized by pattern, with the KPIs the clustering already computes:
how often you buy, what a typical purchase costs, monthly spend, and a short
plain-language description of the habit. It doubles as an explainability view:
you can see exactly which merchants form each pattern and why they belong
together.

## UI

- New section under "Patterns in detail", rendered as custom HTML via
  `st.markdown(unsafe_allow_html=True)`, matching the pattern-card styling
  (same colors, borders, and the donut color per pattern).
- Chrome is minimal: the section header plus one short caption. No long
  explanation paragraph and no legend table of behaviour rules in the UI
  (those existed only in the design mockup).
- One collapsible group per pattern, using native `<details>` and `<summary>`
  elements. No JavaScript, no Streamlit rerun on toggle.
- The `<summary>` header shows: pattern color dot, pattern name, category,
  merchant count, and total monthly spend. All groups start collapsed, so the
  section is only a few lines tall until the user opens one.
- Inside: a table with columns Merchant, Frequency, Avg purchase,
  Monthly spend, Behaviour. Merchants sorted by monthly spend, descending.
  All merchants are shown (no top-N cutoff); the collapse handles length.
- Frequency is formatted for humans, never as a raw rate:
  - 1.5+ per month: "4/month" (rounded)
  - 0.8 to 1.5: "monthly"
  - 0.4 to 0.8: "every 2 months"
  - 0.28 to 0.4: "every 3 months"
  - below: "1-2 times in 6 months"

## Data flow

Everything numeric already exists. `build_pipeline` in `app.py` returns the
per-merchant feature matrix (`features.build_features`: `tx_per_month`,
`avg_amount`, `monthly_spend`, `interval_regularity`, `amount_stability`) and
the cluster labels. The new section joins those with the pattern names from
the analysis result. No new computation beyond formatting.

## Behaviour phrases

Two sources, same contract as cluster names today (AI when available, local
fallback otherwise):

1. **GPT-5.6 (primary).** Extend the existing single `analyze_clusters` call
   in `core/llm.py`; do not add a second API call. Each cluster's stats gain a
   compact merchant feature table: name, tx_per_month, avg_amount,
   interval_regularity, amount_stability, all rounded. This stays within the
   module's privacy promise (aggregates only, never raw transactions; merchant
   names already appear in `top_merchants`). The response schema gains, per
   cluster, `"merchant_behaviours": {merchant: phrase}`.
   - Validation, alongside the existing response validation: each phrase is a
     string of at most 40 characters, no digits (
[truncated — 2270 more characters]
```

### docs/DESIGN.md

```markdown
# AI Personal Finance Assistant — Design Spec

**Date:** 2026-07-15
**Context:** OpenAI Build Week 2026
**Track:** Apps for Your Life
**Submission deadline:** July 21, 2026, 5:00 pm PDT

## What it is

AI Personal Finance Assistant is a personal-finance dashboard for understanding spending behaviour.

A user uploads a common bank transaction export (`.xlsx` or `.csv`). The app cleans and normalizes the data, identifies behavioural spending patterns with K-means clustering, uses GPT-5.6 to name those patterns and explain trends in plain English, and lets the user download a PDF report with their dashboard and summary.

The first release focuses on **expenses only**.

## Product promise

> Upload a bank export and understand the habits behind your spending—not just where the money went.

Examples of outputs:

- “Groceries”
- “Recurring subscriptions”
- “Weekend food delivery”
- “Taxi”
- “Your food-delivery spending increased 28% compared with the previous month.”

## Privacy and data handling

AI Personal Finance Assistant is privacy-conscious, but not fully browser-only in its hosted version.

- The public app runs on Streamlit Community Cloud.
- Uploaded files are processed transiently in the app’s server memory.
- AI Personal Finance Assistant does not save uploads, raw transactions, or reports to a database or disk.
- Users never provide an API key.
- The app’s OpenAI API key is stored privately in Streamlit Community Cloud Secrets.
- GPT-5.6 receives only limited information needed for its tasks:
  - Optional AI-assisted column mapping: headers and up to five sample rows, after explicit user consent.
  - Cluster naming and analysis: aggregated cluster statistics only.
- Users can skip AI-assisted mapping and select their columns manually.
- The public demo and submission video use synthetic data only.

The UI must clearly explain this before upload and before any optional AI-assisted mapping request.

## Supported input

Supported file types:

- `.csv`
- `.xlsx`

AI Personal Finance Assistant supports **common bank exports with guided column mapping**. It does not promise to parse every possible spreadsheet format.

Required fields:

- Transaction date
- Transaction description / merchant
- Amount

Optional fields:

- Bank-provided category
- Account
- Currency

The app supports either one signed amount column or separate debit and credit columns. Only expense transactions are included in the dashboard analysis.

## Stack

- Python 3
- Streamlit
- pandas
- scikit-learn
- Plotly
- OpenAI Python SDK using GPT-5.6
- fpdf2
- kaleido
- pytest
- openpyxl for `.xlsx`

React/Vite is intentionally out of scope: this is a seven-day solo build and the Python data stack is a better fit.

## Architecture

Each module is independent and testable without the Streamlit UI.

```text
parser.py
uploaded file → validated, normalized transaction DataFrame

merchants.py
raw transaction descriptions → cleaned merchant identifiers

features.py
normalized DataFrame → 
[truncated — 10735 more characters]
```

### pyproject.toml

```
[project]
name = "ai-personal-finance-assistant"
version = "0.1.0"
description = "Personal finance spending-pattern dashboard for OpenAI Build Week 2026."
requires-python = ">=3.11,<3.13"
dependencies = [
    "fpdf2>=2.7",
    "kaleido>=0.2.1",
    "openai>=1.40",
    "openpyxl>=3.1",
    "pandas>=2.2",
    "plotly>=5.22",
    "scikit-learn>=1.5",
    "streamlit>=1.36",
]

[dependency-groups]
dev = [
    "pytest>=8.0",
    "watchdog>=6.0.0",
]

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.uv]
package = false

```

### app.py

```python
"""AI Personal Finance Assistant walking skeleton: upload a bank export or explore demo data."""
from __future__ import annotations

import hashlib
import html
import json
import logging
import math
import os
from pathlib import Path

import pandas as pd
import plotly.express as px
import streamlit as st
from streamlit.elements.lib.streamlit_plotly_theme import (
    BG_COLOR,
    CATEGORY_0,
    CATEGORY_1,
    CATEGORY_2,
    CATEGORY_3,
    CATEGORY_4,
    GRAY_90,
)

from core import clustering, features, llm, merchants, parser

logger = logging.getLogger(__name__)

DEMO_PATH = Path(__file__).resolve().parent / "data" / "spending_demo.csv"
LIMIT_ANALYSES = 5
MAX_ROWS = 5000
CURRENCY_MARKERS = {
    "€": ("EUR", "€"),
    "$": ("USD", "$"),
    "£": ("GBP", "£"),
    "¥": ("JPY", "¥"),
}


@st.cache_data(show_spinner="Asking GPT-5.6 for analysis…", ttl="24h")
def cached_analysis(stats_json: str, period_json: str, prompt_fingerprint: str) -> dict:
    """One GPT-5.6 call per unique dataset: identical inputs (any rerun,
    any visitor) reuse the cached result instead of paying for a new call.
    The prompt fingerprint busts stale cache entries whenever the analysis
    prompt changes, so a redeployed prompt takes effect immediately."""
    return llm.analyze_clusters(json.loads(stats_json), json.loads(period_json))


@st.cache_data(show_spinner="Asking GPT-5.6 for a summary…", ttl="24h")
def cached_summary(
    stats_json: str, period_json: str, recurring_json: str, prompt_fingerprint: str
) -> str:
    """One summary call per (dataset, selected period): switching periods
    regenerates the summary for that window once, then serves it from cache."""
    return llm.summarize_period(
        json.loads(stats_json), json.loads(period_json), json.loads(recurring_json)
    )


@st.cache_data(show_spinner="Analyzing your transactions…")
def build_pipeline(raw_df: pd.DataFrame, mapping: parser.ColumnMapping):
    """Parse, normalize, and cluster the full export. (stats schema v2)

    Independent of the live period-filter widget and the theme, so a rerun
    triggered by either (or by anything else) reuses this instead of
    re-parsing and re-clustering from scratch every time. The schema note
    above busts st.cache_data when cluster_stats gains fields, since the
    cache key only hashes this function's own source.
    """
    parse_result = parser.apply_mapping(raw_df, mapping)
    all_transactions = merchants.add_merchant_column(parse_result.df)
    feats = cluster_result = None
    all_stats: list[dict] = []
    if len(all_transactions) >= clustering.MIN_TRANSACTIONS:
        feats = features.build_features(all_transactions)
        categories = all_transactions.groupby("merchant")["category"].agg(
            lambda values: values.mode().iat[0] if not values.mode().empty else ""
        )
        cluster_result = clustering.cluster_merchants(
            feats, n_transactions=len(all_transactions), categories=categories
        )
        all_stats = clustering.cluster_stats(all_transactions, cluster_result.labels)
    return parse_result, all_transactions, feats, cluster_result, all_stats


def infer_currency(raw_df: pd.DataFrame) -> str:
    """Return the most common explicit currency marker from an export."""
    sample = " ".join(raw_df.head(200).astype(str).to_numpy().ravel()).upper()
    matches = {
        symbol: sum(sample.count(marker) for marker in markers)
        for symbol, markers in CURRENCY_MARKERS.items()
    }
    return max(matches, key=matches.get) if any(matches.values()) else ""


def format_amount(value: float, currency: str) -> str:
    """Format an amount without inventing a currency where none was exported."""
    return f"{currency}{value:,.2f}"


DONUT_COLORS = ["#7C3AED", "#A78BFA", "#C4B5FD", "#E9D5FF", "#D946EF"]
DONUT_LABEL_INK = ["#F8F7FF", "#2E1065", "#2E1065", "#2E1065", "#F8F7FF"]


def _donut_point(cx: float, cy: float, r: float, a: float) -> str:
    return f"{cx + r * math.cos(a):.2f} {cy + r * math.sin(a):.2f}"


def _donut_sector(cx, cy, r0, r1, a0, a1, corner):
    """SVG path for an annular sector with rounded corners (d3-style arcs).

    Plotly's pie trace cannot round slice corners, so the donut is drawn as
    plain SVG. The corner radius is clamped so tiny slices shrink their
    rounding instead of degenerating (verified against 3% slices).
    """
    span = a1 - a0
    corner = min(corner, (r1 - r0) / 2, r1 * span * 0.35)
    phi_out, phi_in = corner / r1, corner / r0
    large_out = 1 if span - 2 * phi_out > math.pi else 0
    large_in = 1 if span - 2 * phi_in > math.pi else 0
    p = lambda r, a: _donut_point(cx, cy, r, a)  # noqa: E731
    return (
        f"M {p(r1, a0 + phi_out)} "
        f"A {r1} {r1} 0 {large_out} 1 {p(r1, a1 - phi_out)} "
        f"A {corner} {corner} 0 0 1 {p(r1 - corner, a1)} "
        f"L {p(r0 + corner, a1)} "
        f"A {corner} {corner} 0 0 1 {p(r0, a1 - phi_in)} "
        f"A {r0} {r0} 0 {large_in} 0 {p(r0, a0 + phi_in)} "
        f"A {corner} {corner} 0 0 1 {p(r0 + corner, a0)} "
        f"L {p(r1 - corner, a0)} "
        f"A {corner} {corner} 0 0 1 {p(r1, a0 + phi_out)} Z"
    )


def rounded_donut_svg(stats: list[dict], names: dict[int, str], currency: str) -> str:
    """Render the spending-by-pattern donut as SVG with rounded slice corners.

    Hovering a slice reveals a styled callout next to that slice. It uses the
    active Streamlit theme's surface, border, and text variables, so it reads
    like the native chart hover cards in either light or dark mode.
    """
    # A compact canvas keeps the donut visually centred in its card instead
    # of shrinking it against a very wide SVG viewport.
    view_width = 420.0
    outer, inner = 180.0, 116.0
    total = sum(item["total_spend"] for item in stats) or 1.0
    gap = math.radians(3.2)
    angle = -math.pi / 2
    cx = view_width / 2
    cy = 190.0
    view_height = cy + outer + 60
    tip_width, tip_height = 154.0, 48.0

    slice_parts: 
[truncated — 37240 more characters]
```

### tests/test_merchants.py

```python
import pandas as pd

from core.merchants import add_merchant_column, normalize_merchant


def test_strips_reference_numbers():
    assert normalize_merchant("SUPERMARKET 1234 CITY") == "SUPERMARKET CITY"


def test_same_merchant_variants_collapse():
    a = normalize_merchant("NETFLIX.COM 12/07 REF98765")
    b = normalize_merchant("NETFLIX.COM 15/08 REF11111")
    assert a == b == "NETFLIX COM"


def test_punctuation_and_case():
    assert normalize_merchant("uber *trip-4412") == "UBER TRIP"


def test_keeps_full_rent_a_car_merchant_name():
    assert normalize_merchant("Sixt Rent a Car 2026-04-08") == "SIXT RENT A CAR"


def test_empty_becomes_unknown():
    assert normalize_merchant("  123456  ") == "UNKNOWN"


def test_add_merchant_column_preserves_description():
    df = pd.DataFrame({"description": ["SHOP 99 MAIN ST"]})
    out = add_merchant_column(df)
    assert out["merchant"].iloc[0] == "SHOP MAIN ST"
    assert out["description"].iloc[0] == "SHOP 99 MAIN ST"

```

### core/merchants.py

```python
"""Raw transaction descriptions -> cleaned merchant identifiers.

Deliberately simple (see docs/DESIGN.md): uppercase, drop tokens containing
digits, strip punctuation, keep the first three tokens. The common “Rent a
Car” suffix is retained so car-hire merchants do not appear as truncated names.
No fuzzy matching.
"""
import re

import pandas as pd

_PUNCT = re.compile(r"[*#/\\\-_.,:;!?()\[\]{}'\"@+&]")


def normalize_merchant(description: str) -> str:
    s = _PUNCT.sub(" ", str(description).upper())
    tokens = [t for t in s.split() if not any(c.isdigit() for c in t)]
    # “SIXT RENT A CAR” and “EUROPCAR RENT A CAR” need all four words for a
    # readable dashboard label. Preserve this suffix without broadening the
    # normal three-token rule for unrelated, noisy bank descriptions.
    for index in range(max(0, len(tokens) - 2)):
        if tokens[index : index + 3] == ["RENT", "A", "CAR"]:
            return " ".join(tokens[: index + 3])
    return " ".join(tokens[:3]) or "UNKNOWN"


def add_merchant_column(df: pd.DataFrame) -> pd.DataFrame:
    out = df.copy()
    out["merchant"] = out["description"].map(normalize_merchant)
    return out

```

### tests/conftest.py

```python
import io

import pandas as pd
import pytest


@pytest.fixture
def simple_df():
    """English bank export, signed amounts (expenses negative)."""
    return pd.DataFrame({
        "Date": ["2026-01-05", "2026-01-06", "2026-01-07", "2026-02-05"],
        "Description": ["SUPERMARKET 123", "SALARY JANUARY", "NETFLIX.COM 555", "NETFLIX.COM 556"],
        "Amount": ["-52.30", "2100.00", "-12.99", "-12.99"],
    })


@pytest.fixture
def european_df():
    """German-style export: dd.mm.yyyy dates, 1.234,56 amounts."""
    return pd.DataFrame({
        "Datum": ["05.01.2026", "06.01.2026", "07.01.2026"],
        "Beschreibung": ["EDEKA FILIALE 44", "MIETE JANUAR", "BAHN TICKET 9981"],
        "Betrag": ["-1.234,56", "-800,00", "-49,90"],
    })


@pytest.fixture
def debit_credit_df():
    """Separate debit / credit columns, category present."""
    return pd.DataFrame({
        "Fecha": ["05/01/2026", "06/01/2026", "07/01/2026"],
        "Concepto": ["MERCADONA 8", "NOMINA", "TAXI 4412"],
        "Cargo": ["45.10", "", "9.80"],
        "Abono": ["", "2000.00", ""],
        "Categoria": ["Comida", "Ingresos", "Transporte"],
    })


@pytest.fixture
def malformed_df():
    """Junk rows mixed in: totals row, blank row, bad date."""
    return pd.DataFrame({
        "Date": ["2026-01-05", "TOTAL", "", "2026-01-08", "not a date"],
        "Description": ["SHOP A", "", "", "SHOP B", "SHOP C"],
        "Amount": ["-10.00", "-999.99", "", "-20.00", "-30.00"],
    })


def make_xlsx_bytes(sheets: dict) -> bytes:
    buf = io.BytesIO()
    with pd.ExcelWriter(buf, engine="openpyxl") as writer:
        for name, df in sheets.items():
            df.to_excel(writer, sheet_name=name, index=False)
    return buf.getvalue()

```

### tests/test_clustering.py

```python
import numpy as np
import pandas as pd
import pytest

import core.clustering as cl
from core.clustering import ClusterResult, cluster_merchants, cluster_stats
from core.features import FEATURE_COLUMNS


def _features(n_merchants, seed=42):
    rng = np.random.default_rng(seed)
    # two obvious blobs so real clustering succeeds
    half = n_merchants // 2
    a = rng.normal(0.0, 0.1, size=(half, len(FEATURE_COLUMNS)))
    b = rng.normal(5.0, 0.1, size=(n_merchants - half, len(FEATURE_COLUMNS)))
    data = np.vstack([a, b])
    return pd.DataFrame(data, columns=FEATURE_COLUMNS,
                        index=[f"M{i}" for i in range(n_merchants)])


def test_clusters_two_blobs():
    r = cluster_merchants(_features(12), n_transactions=50)
    assert not r.used_fallback
    assert r.k == 2
    assert r.silhouette > 0.5


def test_fallback_too_few_transactions():
    r = cluster_merchants(_features(12), n_transactions=9)
    assert r.used_fallback and r.reason == "not_enough_transactions"


def test_fallback_too_few_merchants():
    r = cluster_merchants(_features(5), n_transactions=50)
    assert r.used_fallback and r.reason == "not_enough_merchants"


def test_fallback_weak_silhouette(monkeypatch):
    monkeypatch.setattr(cl, "silhouette_score", lambda X, labels: 0.05)
    r = cluster_merchants(_features(12), n_transactions=50)
    assert r.used_fallback and r.reason == "weak_clusters"


def test_fallback_uses_categories_when_available():
    feats = _features(5)
    cats = pd.Series(["Food", "Food", "Transport", "Transport", "Food"],
                     index=feats.index)
    r = cluster_merchants(feats, n_transactions=50, categories=cats)
    assert r.used_fallback
    assert r.labels.nunique() == 2


def test_cluster_stats_aggregates():
    df = pd.DataFrame({
        "date": pd.to_datetime(["2026-01-05", "2026-01-20", "2026-02-05"]),
        "amount": [10.0, 20.0, 12.99],
        "description": ["SHOP A 1", "SHOP A 2", "NETFLIX"],
        "category": ["", "", ""],
        "merchant": ["SHOP A", "SHOP A", "NETFLIX"],
    })
    labels = pd.Series({"SHOP A": 0, "NETFLIX": 1})
    stats = cluster_stats(df, labels)
    c0 = next(s for s in stats if s["cluster_id"] == 0)
    assert c0["n_transactions"] == 2
    assert c0["total_spend"] == pytest.approx(30.0)
    assert c0["top_merchants"] == ["SHOP A"]
    assert "2026-01" in c0["monthly_totals"]

```

### core/features.py

```python
"""Normalized expense DataFrame -> per-merchant feature matrix.

Volume features are normalized per month so different upload periods are
comparable. Recurrence is computed across the whole period (a monthly
subscription is invisible inside a single month).
"""
import numpy as np
import pandas as pd

FEATURE_COLUMNS = ["monthly_spend", "tx_per_month", "avg_amount", "amount_cv",
                   "log_avg_amount", "weekend_share", "interval_regularity",
                   "amount_stability"]


def analysis_period(df: pd.DataFrame):
    """Months = calendar months spanned (Jan 10 - Feb 20 counts as 2), so
    monthly rates aren't inflated for uploads starting mid-month."""
    start, end = df["date"].min(), df["date"].max()
    months = (end.year * 12 + end.month) - (start.year * 12 + start.month) + 1
    return start, end, max(months, 1)


def build_features(df: pd.DataFrame) -> pd.DataFrame:
    _, _, months = analysis_period(df)
    rows = []
    for merchant, g in df.groupby("merchant"):
        amounts, n = g["amount"], len(g)
        avg = float(amounts.mean())
        std = float(amounts.std(ddof=0)) if n > 1 else 0.0
        cv = std / avg if avg > 0 else 0.0
        gaps = g["date"].sort_values().diff().dt.days.dropna()
        if len(gaps) >= 2 and gaps.mean() > 0:
            regularity = 1.0 / (1.0 + float(gaps.std(ddof=0)) / float(gaps.mean()))
        else:
            regularity = 0.0
        rows.append({
            "merchant": merchant,
            "monthly_spend": float(amounts.sum()) / months,
            "tx_per_month": n / months,
            "avg_amount": avg,
            "amount_cv": cv,
            "log_avg_amount": float(np.log1p(avg)),
            "weekend_share": float((g["date"].dt.dayofweek >= 5).mean()),
            "interval_regularity": regularity,
            "amount_stability": 1.0 / (1.0 + cv),
        })
    return pd.DataFrame(rows).set_index("merchant")[FEATURE_COLUMNS]


def recurring_charges(feats: pd.DataFrame) -> list[list]:
    """[merchant, exact monthly price] for every charge with a regular
    monthly rhythm and a stable amount — the candidate subscription list
    handed to the AI summary as given facts (it filters out rent/bills)."""
    mask = (
        (feats["interval_regularity"] >= 0.8)
        & (feats["amount_stability"] >= 0.97)
        & feats["tx_per_month"].between(0.8, 1.3)
    )
    selected = feats[mask].sort_values("avg_amount", ascending=False)
    return [[merchant, round(float(row["avg_amount"]), 2)]
            for merchant, row in selected.iterrows()]


def format_frequency(tx_per_month: float) -> str:
    """Human wording for a purchase rate; never a raw '0.53/month'."""
    if tx_per_month >= 1.5:
        return f"{round(tx_per_month)}/month"
    if tx_per_month >= 0.8:
        return "monthly"
    if tx_per_month >= 0.4:
        return "every 2 months"
    if tx_per_month >= 0.28:
        return "every 3 months"
    return "1-2 times in 6 months"


def behaviour_label(tx_per_month: float, avg_amount: float,
                    interval_regularity: float, amount_stability: float) -> str:
    """Rule-based habit phrase; the local fallback for AI-written behaviours."""
    if tx_per_month >= 3.5:
        return "frequent small purchases" if avg_amount < 15 else "frequent shopping"
    if tx_per_month >= 1.5:
        return "regular shopping" if interval_regularity >= 0.6 else "repeat purchases"
    if 0.8 <= tx_per_month <= 1.3 and interval_regularity >= 0.8:
        if avg_amount >= 100:
            return "regular fixed payment"  # rent-sized, not a "subscription"
        return ("recurring subscription" if amount_stability >= 0.97
                else "steady monthly purchase")
    if 0.4 <= tx_per_month < 0.8 and interval_regularity >= 0.9:
        return "recurring bimonthly bill"
    if 0.4 <= tx_per_month < 0.8 and interval_regularity >= 0.6:
        return "occasional shopping trips"
    if avg_amount >= 100:
        return "occasional big-ticket purchase"
    return "one-off purchases"

```

### core/clustering.py

```python
"""Per-merchant feature matrix -> K-means clusters and aggregate statistics."""
from __future__ import annotations

from dataclasses import dataclass

import pandas as pd
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

MIN_TRANSACTIONS = 10
MIN_MERCHANTS = 6
MIN_SILHOUETTE = 0.15
MAX_K = 5


@dataclass
class ClusterResult:
    """Result of clustering the merchant feature matrix."""

    labels: pd.Series
    k: int
    silhouette: float | None
    used_fallback: bool
    reason: str | None


def _fallback(
    features: pd.DataFrame,
    reason: str,
    categories: pd.Series | None,
) -> ClusterResult:
    """Use provided categories, or one neutral group, when K-means is unsuitable."""
    if categories is not None and categories.replace("", pd.NA).notna().any():
        aligned_categories = categories.reindex(features.index).fillna("")
        # A fallback must respect the same maximum as modelled clusters. Keep the
        # most common categories and group the long tail into one neutral bucket.
        if aligned_categories.nunique() > MAX_K:
            largest_categories = aligned_categories.value_counts().nlargest(MAX_K - 1).index
            aligned_categories = aligned_categories.where(
                aligned_categories.isin(largest_categories), "Other"
            )
        codes, _ = pd.factorize(aligned_categories)
        labels = pd.Series(codes, index=features.index, dtype="int64")
    else:
        labels = pd.Series(0, index=features.index, dtype="int64")

    return ClusterResult(
        labels=labels,
        k=int(labels.nunique()),
        silhouette=None,
        used_fallback=True,
        reason=reason,
    )


def cluster_merchants(
    features: pd.DataFrame,
    n_transactions: int,
    categories: pd.Series | None = None,
) -> ClusterResult:
    """Cluster merchants with silhouette-selected K-means and safe fallbacks."""
    n_merchants = len(features)
    if n_transactions < MIN_TRANSACTIONS:
        return _fallback(features, "not_enough_transactions", categories)
    if n_merchants < MIN_MERCHANTS:
        return _fallback(features, "not_enough_merchants", categories)

    k_max = min(MAX_K, n_merchants // 3)
    if k_max < 2:
        return _fallback(features, "no_valid_k", categories)

    scaled_features = StandardScaler().fit_transform(features.to_numpy())
    best_k: int | None = None
    best_score = -1.0
    best_labels = None

    for k in range(2, k_max + 1):
        model = KMeans(n_clusters=k, n_init=10, random_state=42)
        labels = model.fit_predict(scaled_features)
        score = silhouette_score(scaled_features, labels)
        if score > best_score:
            best_k = k
            best_score = float(score)
            best_labels = labels

    if best_score < MIN_SILHOUETTE or best_k is None or best_labels is None:
        return _fallback(features, "weak_clusters", categories)

    return ClusterResult(
        labels=pd.Series(best_labels, index=features.index, dtype="int64"),
        k=best_k,
        silhouette=best_score,
        used_fallback=False,
        reason=None,
    )


def cluster_stats(df: pd.DataFrame, labels: pd.Series) -> list[dict]:
    """Return explainable, aggregate statistics for each merchant cluster."""
    work = df.assign(cluster=df["merchant"].map(labels)).dropna(subset=["cluster"])
    statistics: list[dict] = []

    for cluster_id, group in work.groupby("cluster"):
        monthly = group.groupby(group["date"].dt.to_period("M"))["amount"].sum()
        top_merchant_spend = group.groupby("merchant")["amount"].sum().nlargest(5)
        statistics.append(
            {
                "cluster_id": int(cluster_id),
                "n_transactions": int(len(group)),
                "n_merchants": int(group["merchant"].nunique()),
                "total_spend": round(float(group["amount"].sum()), 2),
                "avg_amount": round(float(group["amount"].mean()), 2),
                "top_merchants": top_merchant_spend.index.tolist(),
                "top_merchant_items": [
                    [merchant, round(float(amount), 2)]
                    for merchant, amount in top_merchant_spend.items()
                ],
                "example_descriptions": group["description"].drop_duplicates().head(3).tolist(),
                "weekend_share": round(
                    float((group["date"].dt.dayofweek >= 5).mean()), 2
                ),
                "monthly_totals": {
                    str(period): round(float(amount), 2)
                    for period, amount in monthly.items()
                },
            }
        )

    return sorted(statistics, key=lambda item: -item["total_spend"])

```

[6 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]