Project Info
[Y Combinator] Build an Iconic YC Company with AI (1st Place: Guaranteed YC interview 2nd Place: Guaranteed YC Office Hours 3rd Place: Guaranteed YC Office Hours)
aesthetica
Imagine this. You’re walking down the street and someone passes you wearing your exact ideal blazer. The cut, the drape, the color temperature, the silhouette against the light. It’s perfect. By the time you pull out your phone, unlock it, open the camera, and awkwardly try to capture the outfit without being obvious… they’re gone. Even if you get the photo: Reverse image search takes minutes. Results are noisy and generic. You forget why you liked it. The moment doesn’t compound.
Inspiration
disappears. Taste doesn’t accumulate. We built Aesthetica to fix that. What Aesthetica Does Aesthetica is spatial fashion intelligence for the real world. With a single gesture while wearing Meta Ray-Ban glasses, users capture any outfit they see. We're also fully integrated with Poke, the best conversational assistant one could have. Within five seconds, Poke pings you on iMessages, "saw you liked that outfit. here's what we found." The system isolates garments in the live camera feed. It extracts structured, multi-attribute embeddings. It performs reverse visual retrieval across product databases. It updates a persistent, interpretable taste graph with a fully unique, fully comparable Fashion Identity. This is not just visual search. It is a continuously learning taste engine. Aesthetica consists of five core layers: 1. Spatial Capture Layer Using Meta Ray-Ban camera input and gesture triggers, we capture short-frame sequences aligned with user gaze. We perform: Real-time object detection Garment segmentation Human pose estimation for body-part localization This allows us to isolate: Tops Bottoms Outerwear Footwear Accessories and anchor them relative to body geometry. 2. Garment & Body Mapping (Computer Vision Stack) We use CV models to: Segment garments from background Map clothing to anatomical regions Extract silhouette contours Estimate drape and structure features Identify layering relationships We compute structured features including: Silhouette type (structured, relaxed, oversized, tapered) Color palette distributions (dominant + secondary tones) Texture embeddings (wool, satin-like, denim-like, etc.) Pattern detection (solid, plaid, striped, etc.) Formality classification Gender-neutral style archetypes Rather than storing a single opaque embedding vector, we decompose each capture into interpretable attribute nodes. 3. Catalog Engine You upload any photo (outfit, garment, or inspiration). The pipeline: Runs OpenAI-based style analysis on the image (garment name, five style scores, and a short description) Uses OpenAI to generate a shopping query and rationale from that style signal Searches the open web via SerpAPI (e.g. Google Shopping) with the query You get visually similar, purchasable items plus a Poke notification with a link and short opener. 4. Structured Taste Graph This is the core innovation. Instead of just saving products, we maintain a dynamic user-specific style graph. Over time, the system learns: What you consistently notice What you ignore How your taste drifts seasonally Which attributes correlate Your aesthetic identity becomes computationally modeled. 5. Persistent Taste Engine Most fashion AI tools answer: “What is this?” Aesthetica answers: “What does this say about you?” We build: A persistent style embedding A continuously updated attribute distribution A style trajectory over time An interpretable preference surface The more you capture, the more accurate the system becomes. Taste compounds. Gesture-triggered capture pipeline Real-time garment segmentation and body mapping Multi-attribute embedding extraction Vector-based reverse image retrieval Structured style graph engine Real-time product surfacing UI Under-five-second end-to-end flow Segmenting garments in uncontrolled, in-the-wild lighting Handling occlusion and motion blur Building an interpretable preference model instead of a black-box vector Balancing retrieval accuracy with low latency Designing a graph update rule that meaningfully reflects aesthetic evolution The hardest problem was not visual search. It was modeling identity. As spatial computing becomes ambient, commerce must become ambient. When cameras are always available and gestures replace screens, discovery should be frictionless. Aesthetica is building the infrastructure layer for spatial commerce: Real-world capture Structured aesthetic modeling Persistent taste intelligence Instant conversion to commerce Fashion isn’t just what you buy. It’s what you notice. And now, noticing is enough.
Aesthetica Monorepo
Spatial fashion capture and taste intelligence MVP for Meta Ray-Ban smart glasses.
Detailed setup: docs/setup.md
Monorepo Layout
apps/mobile-captureFlutter companion app with DAT abstraction, frame preprocessing, upload.apps/ui-aestheticaNext.js landing page + dashboard (App Router).apps/dashboard-webLegacy React + Vite dashboard (kept for reference).services/apiFastAPI API, auth, capture ingestion, DB models, endpoints.services/workerCelery worker for async inference pipeline.services/mlShared ML pipeline code: segmentation, embeddings, attributes, FAISS, radar.infraDocker Compose for local dev stack.docsArchitecture and API notes.dataDemo catalog, FAISS artifacts, uploads.
Prerequisites
- Docker + Docker Compose
- Python 3.11+ (optional, for local non-docker runs)
- Node 20+ and pnpm/npm (dashboard local run)
- Flutter 3.24+ (mobile app)
Quick Start (Docker)
- Copy env:
cp .env.example .env
- Start infra + services:
make dev
- Run DB migrations and seed demo data:
make migrate
make seed
- Build product embeddings + FAISS indexes:
make embed-products
API: http://localhost:8000
Dashboard: http://localhost:5173
Catalog API: http://localhost:8001
Running Services Individually
API + Worker + DB with Docker Compose
docker compose -f infra/docker-compose.yml up --build
UI (Landing + Dashboard)
make ui
This serves:
- Landing:
http://localhost:5173/ - Dashboard:
http://localhost:5173/dashboard
Mobile Capture (Flutter)
cd apps/mobile-capture
flutter pub get
flutter run
Dev notes for DAT integration:
- The app uses
DatServiceabstraction (MethodChannel) so native DAT SDK integration can be plugged in on iOS/Android. - iOS bridge is now implemented in
apps/mobile-capture/ios/Runner/AppDelegate.swiftwith:- DAT provider path (
MWDATCore+MWDATCamera, when SPM dependency is installed) - AVFoundation fallback stream path (phone camera)
- DAT provider path (
- Hardware camera-button capture is wired through DAT
photoDataPublisherto auto-upload flow. - Capture trigger supports in-app button; extendable for physical button callbacks and volume-button shortcut.
Product Catalog + Indexing
- Source CSV:
data/products.csv - Embedding script:
services/ml/scripts/embed_products.py - FAISS output:
data/faiss/*.indexanddata/faiss/*_mapping.json - Open-web match fallback: SerpAPI Google Shopping
Environment Variables
See .env.example.
Key ones:
DATABASE_URLREDIS_URLFAISS_DIRPRODUCT_CSV_PATHOPENCLIP_MODEL_NAMEOPENCLIP_PRETRAINEDPOKE_API_KEYSERPAPI_API_KEY(for live online-shop search beyond local catalog)
API Docs
- OpenAPI UI:
http://localhost:8000/docs - Human-readable spec:
docs/api.md
Tests
make test
Includes:
- Unit tests for embedding/radar math.
- Integration test for capture pipeline (mock model providers).
Security + Privacy (MVP)
- Stores only cropped/blurred capture image.
- Runs backend safety face blur pass before persistence.
- No full scene frame persisted.
- Basic capture endpoint rate limit.
Operations
- Health:
/healthz,/readyz - JSON structured logs with request/capture correlation IDs.
- Celery async jobs with retry support.
Catalog API (Restored)
Dedicated endpoint for image -> OpenAI -> Serp -> DB write:
POST http://localhost:8001/v1/catalog/from-image- No auth
- Accepts multipart
imageupload or rawimage/jpegbody - Uploads the input image to Supabase Storage bucket
captures(best effort) whenSUPABASE_URLandSUPABASE_SERVICE_ROLE_KEYare set - Runs additional style-recommendation flow immediately:
- OpenAI call #1: style description + 5 scores (0-100) into
style_scores - Aggregate last 5 score rows/descriptions
- OpenAI call #2: recommendation rationale + search query
- One Serp shopping call -> top 5 rows into
style_recommendations
- OpenAI call #1: style description + 5 scores (0-100) into
Run (from repo root):
cp .env.example .env
# set these for storage upload of API input images
# SUPABASE_URL=https://<project-ref>.supabase.co
# SUPABASE_SERVICE_ROLE_KEY=<service-role-key>
docker compose -f infra/docker-compose.yml up -d --build postgres redis api catalog-api
make migrate
Test with any local image file (example below uses one that exists in this repo):
curl -sS -X POST "http://localhost:8001/v1/catalog/from-image" -F "image=@apps/ui-aesthetica/public/images/outfit-1.png"
Verify latest writes in the configured database:
docker compose -f infra/docker-compose.yml run --rm api sh -lc 'python - <<\"PY\"
from sqlalchemy import create_engine, text
from app.core.config import settings
engine = create_engine(settings.database_url)
with engine.connect() as c:
print("catalog_requests:", c.execute(text("select count(*) from catalog_requests")).scalar())
print("style_scores:", c.execute(text("select count(*) from style_scores")).scalar())
print("style_recommendations:", c.execute(text("select count(*) from style_recommendations")).scalar())
PY'
Quick health checks:
curl -s http://localhost:8001/healthz
curl -s http://localhost:8001/readyz
OpenAI -> Shopping Smoke Test (No Integration Suite Required)
This script runs a real image through:
POST /v1/catalog/from-image- Supabase capture upload URL check
- OpenAI image analysis for clothing description + brand/color/style cues
- OpenAI query builder for shopping retrieval
- Serp Google Shopping using the OpenAI-built query
Run from repo root:
docker compose -f infra/docker-compose.yml run --rm api \
python services/api/app/scripts/test_openai_shopping_pipeline.py \
--image apps/ui-aesthetica/public/images/outfit-1.png \
--api-base http://catalog-api:8000
Or use Make:
make test-openai-shopping
# custom image:
make test-openai-shopping OPENAI_TEST_IMAGE=apps/ui-aesthetica/public/images/outfit-9.png
The script prints:
request_idcapture_blob_url+ HTTP statusquery_used(final OpenAI-built query used for shopping)- top Shopping results returned by the API
Analysis
View
Metric
- 18
- 7
- 6
- 5
- 5
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CIn code
- CSSIn code
- DartIn code
- FastAPIIn code
- HTMLIn code
- JavaIn code
- Next.jsIn code
- PythonIn code
- PyTorchIn code
- ReactIn code
- RedisIn code
- SwiftIn code
- Tailwind CSSIn code
- TypeScriptIn code
14 of 14 appear in the indexed code.
AI coding agents
- CursorCommits
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
1.1 MB
Source files
222
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
socratesosorio/aesthetica
314 files · 36.1 MB · @ 8f6279b
Structure
Interface
95 files · 30%Screens, components and styles rendered to the user.
API & routing
48 files · 15%Request entry points: routes, handlers and controllers.
Application logic
60 files · 19%Domain rules, services and shared utilities.
Background jobs
6 files · 2%Work run outside a request: tasks, workers and schedules.
Data & schema
15 files · 5%Schema definitions, migrations and data access.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- TypeScript46%
- Python22%
- YAML14%
- Dart11%
- Markdown3%
- Swift2%
- Other (4)2%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
apps/ui-aesthetica/package.json
npm · 61- @hookform/resolvers
- @radix-ui/react-accordion
- @radix-ui/react-alert-dialog
- @radix-ui/react-aspect-ratio
- @radix-ui/react-avatar
- @radix-ui/react-checkbox
- @radix-ui/react-collapsible
- @radix-ui/react-context-menu
- @radix-ui/react-dialog
- @radix-ui/react-dropdown-menu
- @radix-ui/react-hover-card
- @radix-ui/react-label
- @radix-ui/react-menubar
- @radix-ui/react-navigation-menu
- @radix-ui/react-popover
- @radix-ui/react-progress
- @radix-ui/react-radio-group
- @radix-ui/react-scroll-area
- +43 more
services/api/requirements.txt
pypi · 18- alembic
- bcrypt
- celery
- email-validator
- fastapi
- httpx
- passlib[bcrypt]
- psycopg[binary]
- pydantic-settings
- PyJWT
- pytest
- pytest-asyncio
- python-dotenv
- python-multipart
- redis
- sqlalchemy
- uvicorn[standard]
- websockets
services/ml/requirements.txt
pypi · 11- boto3
- faiss-cpu
- numpy
- open_clip_torch
- opencv-python-headless
- pandas
- pillow
- python-dotenv
- requests
- scikit-learn
- torch
apps/dashboard-web/package.json
npm · 10- d3
- react
- react-dom
- react-router-dom
- +6 more
services/worker/requirements.txt
pypi · 5- celery
- psycopg[binary]
- python-dotenv
- redis
- sqlalchemy
services/poke-mcp/requirements.txt
pypi · 3- fastmcp
- httpx
- requests
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.