# Project export: aesthetica This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2026 - Tagline: the 1st spatial fashion system. w 1 gesture on meta glasses, u turn any outfit u c into aesthetic data, matched products, a taste profile that evolves w u u see it u like it u want it u got it. - Devpost: https://devpost.com/software/aesthetica - GitHub: https://github.com/socratesosorio/aesthetica - Demo: https://aesthetica-ai.vercel.app/ - Video: https://www.youtube.com/embed/ZhqUDfSZVuw?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner ([Y Combinator] Build an Iconic YC Company with AI (1st Place: Guaranteed YC interview 2nd Place: Guaranteed YC Office Hours 3rd Place: Guaranteed YC Office Hours)) - Team: 5 GitHub contributor(s) — Socrates Osorio Diaz (18 commits), Anthony-Tafoya (7 commits), Joy Yang (6 commits), Cursor (5 commits), Tevin Wang (5 commits) ## Devpost submission (written by the team) ### Overview Imagine this. You’re walking down the street and someone passes you wearing your exact ideal blazer. The cut, the drape, the color temperature, the silhouette against the light. It’s perfect. By the time you pull out your phone, unlock it, open the camera, and awkwardly try to capture the outfit without being obvious… they’re gone. Even if you get the photo: Reverse image search takes minutes. Results are noisy and generic. You forget why you liked it. The moment doesn’t compound. ### Inspiration disappears. Taste doesn’t accumulate. We built Aesthetica to fix that. What Aesthetica Does Aesthetica is spatial fashion intelligence for the real world. With a single gesture while wearing Meta Ray-Ban glasses, users capture any outfit they see. We're also fully integrated with Poke, the best conversational assistant one could have. Within five seconds, Poke pings you on iMessages, "saw you liked that outfit. here's what we found." The system isolates garments in the live camera feed. It extracts structured, multi-attribute embeddings. It performs reverse visual retrieval across product databases. It updates a persistent, interpretable taste graph with a fully unique, fully comparable Fashion Identity. This is not just visual search. It is a continuously learning taste engine. Aesthetica consists of five core layers: 1. Spatial Capture Layer Using Meta Ray-Ban camera input and gesture triggers, we capture short-frame sequences aligned with user gaze. We perform: Real-time object detection Garment segmentation Human pose estimation for body-part localization This allows us to isolate: Tops Bottoms Outerwear Footwear Accessories and anchor them relative to body geometry. 2. Garment & Body Mapping (Computer Vision Stack) We use CV models to: Segment garments from background Map clothing to anatomical regions Extract silhouette contours Estimate drape and structure features Identify layering relationships We compute structured features including: Silhouette type (structured, relaxed, oversized, tapered) Color palette distributions (dominant + secondary tones) Texture embeddings (wool, satin-like, denim-like, etc.) Pattern detection (solid, plaid, striped, etc.) Formality classification Gender-neutral style archetypes Rather than storing a single opaque embedding vector, we decompose each capture into interpretable attribute nodes. 3. Catalog Engine You upload any photo (outfit, garment, or inspiration). The pipeline: Runs OpenAI-based style analysis on the image (garment name, five style scores, and a short description) Uses OpenAI to generate a shopping query and rationale from that style signal Searches the open web via SerpAPI (e.g. Google Shopping) with the query You get visually similar, purchasable items plus a Poke notification with a link and short opener. 4. Structured Taste Graph This is the core innovation. Instead of just saving products, we maintain a dynamic user-specific style graph. Over time, the system learns: What you consistently notice What you ignore How your taste drifts seasonally Which attributes correlate Your aesthetic identity becomes computationally modeled. 5. Persistent Taste Engine Most fashion AI tools answer: “What is this?” Aesthetica answers: “What does this say about you?” We build: A persistent style embedding A continuously updated attribute distribution A style trajectory over time An interpretable preference surface The more you capture, the more accurate the system becomes. Taste compounds. Gesture-triggered capture pipeline Real-time garment segmentation and body mapping Multi-attribute embedding extraction Vector-based reverse image retrieval Structured style graph engine Real-time product surfacing UI Under-five-second end-to-end flow Segmenting garments in uncontrolled, in-the-wild lighting Handling occlusion and motion blur Building an interpretable preference model instead of a black-box vector Balancing retrieval accuracy with low latency Designing a graph update rule that meaningfully reflects aesthetic evolution The hardest problem was not visual search. It was modeling identity. As spatial computing becomes ambient, commerce must become ambient. When cameras are always available and gestures replace screens, discovery should be frictionless. Aesthetica is building the infrastructure layer for spatial commerce: Real-world capture Structured aesthetic modeling Persistent taste intelligence Instant conversion to commerce Fashion isn’t just what you buy. It’s what you notice. And now, noticing is enough. ## README (from the GitHub repository) # Aesthetica Monorepo Spatial fashion capture and taste intelligence MVP for Meta Ray-Ban smart glasses. Detailed setup: `docs/setup.md` ## Monorepo Layout - `apps/mobile-capture` Flutter companion app with DAT abstraction, frame preprocessing, upload. - `apps/ui-aesthetica` Next.js landing page + dashboard (App Router). - `apps/dashboard-web` Legacy React + Vite dashboard (kept for reference). - `services/api` FastAPI API, auth, capture ingestion, DB models, endpoints. - `services/worker` Celery worker for async inference pipeline. - `services/ml` Shared ML pipeline code: segmentation, embeddings, attributes, FAISS, radar. - `infra` Docker Compose for local dev stack. - `docs` Architecture and API notes. - `data` Demo catalog, FAISS artifacts, uploads. ## Prerequisites - Docker + Docker Compose - Python 3.11+ (optional, for local non-docker runs) - Node 20+ and pnpm/npm (dashboard local run) - Flutter 3.24+ (mobile app) ## Quick Start (Docker) 1. Copy env: ```bash cp .env.example .env ``` 2. Start infra + services: ```bash make dev ``` 3. Run DB migrations and seed demo data: ```bash make migrate make seed ``` 4. Build product embeddings + FAISS indexes: ```bash make embed-products ``` API: `http://localhost:8000` Dashboard: `http://localhost:5173` Catalog API: `http://localhost:8001` ## Running Services Individually ### API + Worker + DB with Docker Compose ```bash docker compose -f infra/docker-compose.yml up --build ``` ### UI (Landing + Dashboard) ```bash make ui ``` This serves: - Landing: `http://localhost:5173/` - Dashboard: `http://localhost:5173/dashboard` ### Mobile Capture (Flutter) ```bash cd apps/mobile-capture flutter pub get flutter run ``` Dev notes for DAT integration: - The app uses `DatService` abstraction (`MethodChannel`) so native DAT SDK integration can be plugged in on iOS/Android. - iOS bridge is now implemented in `apps/mobile-capture/ios/Runner/AppDelegate.swift` with: - DAT provider path (`MWDATCore` + `MWDATCamera`, when SPM dependency is installed) - AVFoundation fallback stream path (phone camera) - Hardware camera-button capture is wired through DAT `photoDataPublisher` to auto-upload flow. - Capture trigger supports in-app button; extendable for physical button callbacks and volume-button shortcut. ## Product Catalog + Indexing - Source CSV: `data/products.csv` - Embedding script: `services/ml/scripts/embed_products.py` - FAISS output: `data/faiss/*.index` and `data/faiss/*_mapping.json` - Open-web match fallback: SerpAPI Google Shopping ## Environment Variables See `.env.example`. Key ones: - `DATABASE_URL` - `REDIS_URL` - `FAISS_DIR` - `PRODUCT_CSV_PATH` - `OPENCLIP_MODEL_NAME` - `OPENCLIP_PRETRAINED` - `POKE_API_KEY` - `SERPAPI_API_KEY` (for live online-shop search beyond local catalog) ## API Docs - OpenAPI UI: `http://localhost:8000/docs` - Human-readable spec: `docs/api.md` ## Tests ```bash make test ``` Includes: - Unit tests for embedding/radar math. - Integration test for capture pipeline (mock model providers). ## Security + Privacy (MVP) - Stores only cropped/blurred capture image. - Runs backend safety face blur pass before persistence. - No full scene frame persisted. - Basic capture endpoint rate limit. ## Operations - Health: `/healthz`, `/readyz` - JSON structured logs with request/capture correlation IDs. - Celery async jobs with retry support. ## Catalog API (Restored) Dedicated endpoint for image -> OpenAI -> Serp -> DB write: - `POST http://localhost:8001/v1/catalog/from-image` - No auth - Accepts multipart `image` upload or raw `image/jpeg` body - Uploads the input image to Supabase Storage bucket `captures` (best effort) when `SUPABASE_URL` and `SUPABASE_SERVICE_ROLE_KEY` are set - Runs additional style-recommendation flow immediately: - OpenAI call #1: style description + 5 scores (0-100) into `style_scores` - Aggregate last 5 score rows/descriptions - OpenAI call #2: recommendation rationale + search query - One Serp shopping call -> top 5 rows into `style_recommendations` Run (from repo root): ```bash cp .env.example .env # set these for storage upload of API input images # SUPABASE_URL=https://.supabase.co # SUPABASE_SERVICE_ROLE_KEY= docker compose -f infra/docker-compose.yml up -d --build postgres redis api catalog-api make migrate ``` Test with any local image file (example below uses one that exists in this repo): ```bash curl -sS -X POST "http://localhost:8001/v1/catalog/from-image" -F "image=@apps/ui-aesthetica/public/images/outfit-1.png" ``` Verify latest writes in the configured database: ```bash docker compose -f infra/docker-compose.yml run --rm api sh -lc 'python - <<\"PY\" from sqlalchemy import create_engine, text from app.core.config import settings engine = create_engine(settings.database_url) with engine.connect() as c: print("catalog_requests:", c.execute(text("select count(*) from catalog_requests")).scalar()) print("style_scores:", c.execute(text("select count(*) from style_scores")).scalar()) print("style_recommendations:", c.execute(text("select count(*) from style_recommendations")).scalar()) PY' ``` Quick health checks: ```bash curl -s http://localhost:8001/healthz curl -s http://localhost:8001/readyz ``` ### OpenAI -> Shopping Smoke Test (No Integration Suite Required) This script runs a real image through: 1. `POST /v1/catalog/from-image` 2. Supabase capture upload URL check 3. OpenAI image analysis for clothing description + brand/color/style cues 4. OpenAI query builder for shopping retrieval 5. Serp Google Shopping using the OpenAI-built query Run from repo root: ```bash docker compose -f infra/docker-compose.yml run --rm api \ python services/api/app/scripts/test_openai_shopping_pipeline.py \ --image apps/ui-aesthetica/public/images/outfit-1.png \ --api-base http://catalog-api:8000 ``` Or use Make: ```bash make test-openai-shopping # custom image: make test-openai-shopping OPENAI_TEST_IMAGE=apps/ui-aesthetica/public/images/outfit-9.png ``` The script prints: - `request_id` - `capture_blob_url` + HTTP status - `query_used` (final OpenAI-built query used for shopping) - top Shopping results returned by the API ## Detected evidence (automated analysis) Indexed codebase: 222 recognized source files, 1101 KB. - C (language) — detected in the code - CSS (language) — detected in the code - Dart (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - Java (language) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - PyTorch (technology) — detected in the code - React (technology) — detected in the code - Redis (technology) — detected in the code - Swift (language) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - AI coding agent: Cursor — evidence: commit authorship or trailers ## Codebase structure (from repository index) ### Files (120 of 274) ``` .dockerignore .env.example .gitignore .vscode/settings.json apps/dashboard-web/index.html apps/dashboard-web/package.json apps/dashboard-web/src/api/client.ts apps/dashboard-web/src/App.tsx apps/dashboard-web/src/components/Layout.tsx apps/dashboard-web/src/components/RadarChart.tsx apps/dashboard-web/src/lib/auth.ts apps/dashboard-web/src/main.tsx apps/dashboard-web/src/pages/AnalyticsPage.tsx apps/dashboard-web/src/pages/LoginPage.tsx apps/dashboard-web/src/pages/LookDetailPage.tsx apps/dashboard-web/src/pages/LooksPage.tsx apps/dashboard-web/src/pages/ProfilePage.tsx apps/dashboard-web/src/styles.css apps/dashboard-web/src/types/api.ts apps/dashboard-web/tsconfig.json apps/dashboard-web/tsconfig.node.json apps/dashboard-web/vite.config.ts apps/mobile-capture/.gitignore apps/mobile-capture/.metadata apps/mobile-capture/analysis_options.yaml apps/mobile-capture/android/.gitkeep apps/mobile-capture/android/app/src/main/java/io/flutter/plugins/GeneratedPluginRegistrant.java apps/mobile-capture/ios/.gitignore apps/mobile-capture/ios/.gitkeep apps/mobile-capture/ios/Flutter/AppFrameworkInfo.plist apps/mobile-capture/ios/Flutter/Debug.xcconfig apps/mobile-capture/ios/Flutter/Release.xcconfig apps/mobile-capture/ios/Podfile apps/mobile-capture/ios/Podfile.lock apps/mobile-capture/ios/Runner.xcodeproj/project.pbxproj apps/mobile-capture/ios/Runner.xcodeproj/project.xcworkspace/contents.xcworkspacedata apps/mobile-capture/ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist apps/mobile-capture/ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved apps/mobile-capture/ios/Runner.xcodeproj/project.xcworkspace/xcshareddata/WorkspaceSettings.xcsettings apps/mobile-capture/ios/Runner.xcodeproj/xcshareddata/xcschemes/Runner.xcscheme apps/mobile-capture/ios/Runner.xcworkspace/contents.xcworkspacedata apps/mobile-capture/ios/Runner.xcworkspace/xcshareddata/swiftpm/Package.resolved apps/mobile-capture/ios/Runner/AppDelegate.swift apps/mobile-capture/ios/Runner/Assets.xcassets/AppIcon.appiconset/Contents.json apps/mobile-capture/ios/Runner/Assets.xcassets/LaunchImage.imageset/Contents.json apps/mobile-capture/ios/Runner/Assets.xcassets/LaunchImage.imageset/README.md apps/mobile-capture/ios/Runner/Base.lproj/LaunchScreen.storyboard apps/mobile-capture/ios/Runner/Base.lproj/Main.storyboard apps/mobile-capture/ios/Runner/Info.plist apps/mobile-capture/ios/Runner/Runner-Bridging-Header.h apps/mobile-capture/ios/RunnerTests/RunnerTests.swift apps/mobile-capture/lib/main.dart apps/mobile-capture/lib/models/capture_state.dart apps/mobile-capture/lib/models/catalog_result.dart apps/mobile-capture/lib/screens/capture_screen.dart apps/mobile-capture/lib/screens/stream_screen.dart apps/mobile-capture/lib/services/api_client.dart apps/mobile-capture/lib/services/capture_preprocessor.dart apps/mobile-capture/lib/services/dat_service.dart apps/mobile-capture/lib/services/mock_dat_service.dart apps/mobile-capture/lib/services/snap_detector_service.dart apps/mobile-capture/lib/services/stream_relay_service.dart apps/mobile-capture/lib/widgets/glass_container.dart apps/mobile-capture/lib/widgets/reticle_overlay.dart apps/mobile-capture/pubspec.lock apps/mobile-capture/pubspec.yaml apps/mobile-capture/README.md apps/ui-aesthetica/.gitignore apps/ui-aesthetica/app/api/image-proxy/route.ts apps/ui-aesthetica/app/api/style-essay/route.ts apps/ui-aesthetica/app/dashboard/page.tsx apps/ui-aesthetica/app/database/page.tsx apps/ui-aesthetica/app/globals.css apps/ui-aesthetica/app/layout.tsx apps/ui-aesthetica/app/login/login-client.tsx apps/ui-aesthetica/app/login/page.tsx apps/ui-aesthetica/app/logout/page.tsx apps/ui-aesthetica/app/page.tsx apps/ui-aesthetica/app/profile/page.tsx apps/ui-aesthetica/components.json apps/ui-aesthetica/components/aesthetica/fashion-identity-tab.tsx apps/ui-aesthetica/components/aesthetica/taste-analytics.tsx apps/ui-aesthetica/components/aesthetica/taste-radar.tsx apps/ui-aesthetica/components/dashboard/cv-body-boxes.tsx apps/ui-aesthetica/components/fade-image.tsx apps/ui-aesthetica/components/header.tsx apps/ui-aesthetica/components/sections/conversion-text-section.tsx apps/ui-aesthetica/components/sections/editorial-section.tsx apps/ui-aesthetica/components/sections/featured-products-section.tsx apps/ui-aesthetica/components/sections/footer-section.tsx apps/ui-aesthetica/components/sections/hero-section.tsx apps/ui-aesthetica/components/sections/outfit-gallery-section.tsx apps/ui-aesthetica/components/sections/philosophy-section.tsx apps/ui-aesthetica/components/sections/technology-section.tsx apps/ui-aesthetica/components/sections/testimonials-section.tsx apps/ui-aesthetica/components/theme-provider.tsx apps/ui-aesthetica/components/ui/accordion.tsx apps/ui-aesthetica/components/ui/alert-dialog.tsx apps/ui-aesthetica/components/ui/alert.tsx apps/ui-aesthetica/components/ui/aspect-ratio.tsx apps/ui-aesthetica/components/ui/avatar.tsx apps/ui-aesthetica/components/ui/badge.tsx apps/ui-aesthetica/components/ui/breadcrumb.tsx apps/ui-aesthetica/components/ui/button-group.tsx apps/ui-aesthetica/components/ui/button.tsx apps/ui-aesthetica/components/ui/calendar.tsx apps/ui-aesthetica/components/ui/card.tsx apps/ui-aesthetica/components/ui/carousel.tsx apps/ui-aesthetica/components/ui/chart.tsx apps/ui-aesthetica/components/ui/checkbox.tsx apps/ui-aesthetica/components/ui/collapsible.tsx apps/ui-aesthetica/components/ui/command.tsx apps/ui-aesthetica/components/ui/context-menu.tsx apps/ui-aesthetica/components/ui/dialog.tsx apps/ui-aesthetica/components/ui/drawer.tsx apps/ui-aesthetica/components/ui/dropdown-menu.tsx apps/ui-aesthetica/components/ui/empty.tsx apps/ui-aesthetica/components/ui/field.tsx apps/ui-aesthetica/components/ui/form.tsx apps/ui-aesthetica/components/ui/hover-card.tsx [154 more files omitted for size] ``` ### Dependencies - apps/dashboard-web/package.json: @types/d3@^7.4.3, @types/react@^18.3.5, @types/react-dom@^18.3.0, @vitejs/plugin-react@^4.3.1, d3@^7.9.0, react@^18.3.1, react-dom@^18.3.1, react-router-dom@^6.26.2, typescript@^5.5.4, vite@^5.4.2 - apps/ui-aesthetica/package.json: @hookform/resolvers@^3.10.0, @radix-ui/react-accordion@1.2.2, @radix-ui/react-alert-dialog@1.1.4, @radix-ui/react-aspect-ratio@1.1.1, @radix-ui/react-avatar@1.1.2, @radix-ui/react-checkbox@1.1.3, @radix-ui/react-collapsible@1.1.2, @radix-ui/react-context-menu@2.2.4, @radix-ui/react-dialog@1.1.4, @radix-ui/react-dropdown-menu@2.1.4, @radix-ui/react-hover-card@1.1.4, @radix-ui/react-label@2.1.1, @radix-ui/react-menubar@1.1.4, @radix-ui/react-navigation-menu@1.2.3, @radix-ui/react-popover@1.1.4, @radix-ui/react-progress@1.1.1, @radix-ui/react-radio-group@1.2.2, @radix-ui/react-scroll-area@1.2.2, @radix-ui/react-select@2.1.4, @radix-ui/react-separator@1.1.1, @radix-ui/react-slider@1.2.2, @radix-ui/react-slot@1.1.1, @radix-ui/react-switch@1.1.2, @radix-ui/react-tabs@1.1.2, @radix-ui/react-toast@1.2.4, @radix-ui/react-toggle@1.1.1, @radix-ui/react-toggle-group@1.1.1, @radix-ui/react-tooltip@1.1.6, @tailwindcss/postcss@^4.1.9, @tensorflow-models/body-pix@^2.2.1, @tensorflow-models/pose-detection@^2.1.3, @tensorflow/tfjs@^4.22.0, @types/node@^22, @types/react@^19, @types/react-dom@^19, @vercel/analytics@1.3.1, autoprefixer@^10.4.20, class-variance-authority@^0.7.1, clsx@^2.1.1, cmdk@1.0.4, date-fns@4.1.0, embla-carousel-react@8.5.1, input-otp@1.4.1, lucide-react@^0.454.0, next@16.0.10, next-themes@^0.4.6, postcss@^8.5, react@19.2.0, react-day-picker@9.8.0, react-dom@19.2.0, react-hook-form@^7.60.0, react-resizable-panels@^2.1.7, recharts@2.15.4, sonner@^1.7.4, tailwind-merge@^3.3.1, tailwindcss@^4.1.9, tailwindcss-animate@^1.0.7, tw-animate-css@1.3.3, typescript@^5, vaul@^1.1.2, zod@3.25.76 - services/api/requirements.txt: alembic@==1.13.2, bcrypt@==3.2.2, celery@==5.4.0, email-validator, fastapi@==0.114.2, httpx@==0.27.2, passlib[bcrypt]@==1.7.4, psycopg[binary]@==3.2.1, pydantic-settings@==2.5.2, PyJWT@==2.9.0, pytest@==8.3.2, pytest-asyncio@==0.24.0, python-dotenv@==1.0.1, python-multipart@==0.0.9, redis@==5.0.8, sqlalchemy@==2.0.34, uvicorn[standard]@==0.30.6, websockets@==13.1 - services/ml/requirements.txt: boto3@==1.35.50, faiss-cpu@==1.8.0.post1, numpy@==1.26.4, open_clip_torch@==2.26.1, opencv-python-headless@==4.10.0.84, pandas@==2.2.2, pillow@==10.4.0, python-dotenv@==1.0.1, requests@==2.32.3, scikit-learn@==1.5.1, torch@==2.4.1 - services/poke-mcp/requirements.txt: fastmcp@>=2.0, httpx, requests - services/worker/requirements.txt: celery@==5.4.0, psycopg[binary]@==3.2.1, python-dotenv@==1.0.1, redis@==5.0.8, sqlalchemy@==2.0.34 ### Recent commits (newest first) - remove cv layer - fix ui - Merge branch 'main' of https://github.com/socratesosorio/aesthetica - deployment config - chore(deploy): remove localhost API defaults - feat(profile): revamp fashion identity visuals - mcp more robust - Merge branch 'main' of https://github.com/socratesosorio/aesthetica - Updated - mcp works - message - Merrrrrrrr - Merge branch 'main' of https://github.com/socratesosorio/aesthetica - new catalog from image - fix(cv): require full-body confidence for labels - Merge branch 'main' of https://github.com/socratesosorio/aesthetica - add poke link - New algo drop - Merge branch 'main' of https://github.com/socratesosorio/aesthetica - mcp works ## Key source files (fetched from GitHub, selected and truncated for size) ### PROMPT.md ```markdown You are a principal engineer + ML engineer + product engineer. You will generate a complete, runnable monorepo implementing the MVP described below, with production-grade structure, clear abstractions, tests, and local-dev Docker Compose. PROJECT NAME Aesthetica — Spatial Fashion Capture & Taste Intelligence for Meta Ray-Ban smart glasses PRIMARY OUTCOME Build an end-to-end system where: 1) A user captures an outfit from Meta Ray-Ban smart glasses (via Meta Wearables Device Access Toolkit “DAT” integrated into a companion mobile app). 2) The mobile app crops/filters and uploads an image frame to a FastAPI backend. 3) The backend performs: - human/clothing parsing (segmentation into garment regions) - CLIP-style embeddings (global + per-garment) - attribute extraction (color, minimalism, structure, formality, silhouette, pattern) - product similarity search over a pre-embedded product catalog (FAISS) - user taste profile updates (EMA embedding + interpretable 5-axis radar) 4) The system sends a result summary + links via Poke messaging. 5) The user can view saved looks + matches + taste radar + history via a mobile-first web dashboard. IMPORTANT REAL-WORLD SDK CONSTRAINTS (DO NOT IGNORE) - Meta Wearables Device Access Toolkit (DAT) is for mobile apps (iOS/Android) connecting to Meta AI glasses sensors (camera). The Meta AI app acts as the bridge for pairing/permissions. - Current DAT preview constraints to respect: - Camera access is available; microphone/speaker via standard Bluetooth profiles. - No Meta Neural Band sensor access in preview. - Custom gesture controls like taps/swipes aren’t offered; only standard events (pause/resume/stop) may be available. - “Hey Meta” custom Meta AI voice command integration is NOT provided; but you can access device microphone to build your own voice commands in-app. - Video stream is constrained (Bluetooth); plan for max ~720p/30fps stream, and adaptive degradation. - Publishing integrations may be gated; still build as a functional dev/test MVP. - Therefore: Implement capture trigger using one of these MVP-safe options: (A) In-app “Capture” button that requests a frame from the glasses stream. (B) Physical glasses capture button if stream exposes photo capture callbacks. (C) Optional: phone volume button as a capture shortcut. Provide clean abstraction so gesture/wristband triggers can be added later without refactor. PRD REQUIREMENTS (MVP SCOPE) Included: - Companion mobile capture app with DAT integration - Reticle-centered crop (fixed bounding box) + downscale + face blur (privacy) - Backend inference pipeline: - segmentation / garment parsing - embeddings (global + per garment) - attribute extraction - product vector search (FAISS) - taste profile engine (EMA embedding + 5D radar) - Poke integration: send user message after processing - Mobile web dashboard: gallery of looks, look detail with matches, taste radar chart, history timeline, simple analytics [truncated — 13795 more characters] ``` ### docs/api.md ```markdown # API Spec (Human-Readable) Base URL: `http://localhost:8000` OpenAPI docs: `/docs` ## Auth - `POST /v1/auth/login` - body: `{ "email": "...", "password": "..." }` - returns: `{ "access_token": "...", "token_type": "bearer" }` - `GET /v1/auth/me` ## Capture - `POST /v1/captures` - multipart field: `image` - returns: `{ "capture_id": "...", "status": "queued" }` - `GET /v1/captures/{capture_id}` - `GET /v1/users/{user_id}/captures?limit=...` ## Results - `GET /v1/users/{user_id}/profile` - `GET /v1/users/{user_id}/radar/history?days=...` - `GET /v1/products/search?embedding_b64=...&garment_type=top` - `GET /v1/products/search?capture_id=...&garment_type=top` - `GET /v1/products/search?capture_id=...&garment_type=top&include_web=true` - `include_web=true` appends live web matches (SerpAPI provider) to catalog matches. ## Internal - `POST /v1/internal/reindex-products` (`Authorization: Bearer dev`) - `POST /v1/internal/recompute-radar` (`Authorization: Bearer dev`) ## Health - `GET /healthz` - `GET /readyz` ## Media - `GET /v1/media?path=...&token=...` - token can also be passed as bearer header. ``` ### infra/docker-compose.yml ```yaml version: '3.9' services: postgres: image: postgres:16 container_name: aesthetica-postgres environment: POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres POSTGRES_DB: aesthetica ports: - "5432:5432" volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U postgres -d aesthetica"] interval: 5s timeout: 3s retries: 20 redis: image: redis:7 container_name: aesthetica-redis ports: - "6379:6379" healthcheck: test: ["CMD", "redis-cli", "ping"] interval: 5s timeout: 3s retries: 20 minio: image: minio/minio:latest container_name: aesthetica-minio command: server /data --console-address ":9001" environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: minioadmin ports: - "9000:9000" - "9001:9001" volumes: - minio_data:/data api: build: context: .. dockerfile: services/api/Dockerfile container_name: aesthetica-api env_file: - ${ENV_FILE:-../.env} environment: PYTHONPATH: /app/services/api:/app/services/ml:/app/services/worker REDIS_URL: redis://redis:6379/0 CELERY_BROKER_URL: redis://redis:6379/0 CELERY_RESULT_BACKEND: redis://redis:6379/1 LOCAL_STORAGE_ROOT: /app/data/uploads FAISS_DIR: /app/data/faiss PRODUCT_CSV_PATH: /app/data/products.csv depends_on: postgres: condition: service_healthy redis: condition: service_healthy ports: - "8000:8000" volumes: - ..:/app command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"] catalog-api: build: context: .. dockerfile: services/api/Dockerfile container_name: aesthetica-catalog-api env_file: - ${ENV_FILE:-../.env} environment: PYTHONPATH: /app/services/api:/app/services/ml:/app/services/worker LOCAL_STORAGE_ROOT: /app/data/uploads depends_on: postgres: condition: service_healthy ports: - "8001:8000" volumes: - ..:/app command: ["uvicorn", "app.catalog_main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"] worker: build: context: .. dockerfile: services/worker/Dockerfile container_name: aesthetica-worker env_file: - ${ENV_FILE:-../.env} environment: PYTHONPATH: /app/services/api:/app/services/ml:/app/services/worker REDIS_URL: redis://redis:6379/0 CELERY_BROKER_URL: redis://redis:6379/0 CELERY_RESULT_BACKEND: redis://redis:6379/1 LOCAL_STORAGE_ROOT: /app/data/uploads FAISS_DIR: /app/data/faiss PRODUCT_CSV_PATH: /app/data/products.csv depends_on: postgres: condition: service_healthy redis: condition: service_healthy volumes: - ..:/app command: ["celery", "-A", "worker.tasks", "worker", "--loglevel=INFO", "-Q", "captures"] poke-mcp: build: context: .. dockerfile: services/poke-mcp/Dockerfile container_name: aesthetica-poke-mcp env_file: - ${ENV_FILE:-../.env} environment: PYTHONPATH: /app/services/api:/app/services/ml:/app/services/poke-mcp DATABASE_URL: postgresql+psycopg://postgres:postgres@postgres:5432/aesthetica REDIS_URL: redis://redis:6379/0 LOCAL_STORAGE_ROOT: /app/data/uploads FAISS_DIR: /app/data/faiss PRODUCT_CSV_PATH: /app/data/products.csv depends_on: postgres: condition: service_healthy ports: - "${POKE_MCP_PORT:-8787}:8787" volumes: - ..:/app command: ["python", "services/poke-mcp/server.py"] volumes: pgdata: minio_data: ``` ### services/poke-mcp/requirements.txt ``` fastmcp>=2.0 httpx requests ``` ### services/worker/requirements.txt ``` celery==5.4.0 redis==5.0.8 sqlalchemy==2.0.34 psycopg[binary]==3.2.1 python-dotenv==1.0.1 ``` ### services/ml/requirements.txt ``` numpy==1.26.4 pillow==10.4.0 opencv-python-headless==4.10.0.84 faiss-cpu==1.8.0.post1 torch==2.4.1 open_clip_torch==2.26.1 pandas==2.2.2 scikit-learn==1.5.1 requests==2.32.3 python-dotenv==1.0.1 boto3==1.35.50 ``` ### services/api/requirements.txt ``` fastapi==0.114.2 uvicorn[standard]==0.30.6 websockets==13.1 sqlalchemy==2.0.34 alembic==1.13.2 psycopg[binary]==3.2.1 python-multipart==0.0.9 pydantic-settings==2.5.2 email-validator==2.2.0 passlib[bcrypt]==1.7.4 bcrypt==3.2.2 PyJWT==2.9.0 redis==5.0.8 celery==5.4.0 httpx==0.27.2 python-dotenv==1.0.1 email-validator pytest==8.3.2 pytest-asyncio==0.24.0 ``` ### services/api/Dockerfile ``` FROM python:3.11-slim ENV PYTHONDONTWRITEBYTECODE=1 \ PYTHONUNBUFFERED=1 WORKDIR /app COPY services/api/requirements.txt /tmp/api-requirements.txt COPY services/ml/requirements.txt /tmp/ml-requirements.txt RUN pip install --no-cache-dir -r /tmp/api-requirements.txt -r /tmp/ml-requirements.txt COPY . /app ENV PYTHONPATH=/app/services/api:/app/services/ml:/app/services/worker CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] ``` ### services/worker/Dockerfile ``` FROM python:3.11-slim ENV PYTHONDONTWRITEBYTECODE=1 \ PYTHONUNBUFFERED=1 WORKDIR /app COPY services/api/requirements.txt /tmp/api-requirements.txt COPY services/ml/requirements.txt /tmp/ml-requirements.txt COPY services/worker/requirements.txt /tmp/worker-requirements.txt RUN pip install --no-cache-dir -r /tmp/api-requirements.txt -r /tmp/ml-requirements.txt -r /tmp/worker-requirements.txt COPY . /app ENV PYTHONPATH=/app/services/api:/app/services/ml:/app/services/worker CMD ["celery", "-A", "worker.tasks", "worker", "--loglevel=INFO", "-Q", "captures"] ``` ### apps/dashboard-web/package.json ``` { "name": "aesthetica-dashboard-web", "private": true, "version": "0.1.0", "type": "module", "scripts": { "dev": "vite --host 0.0.0.0 --port 5173", "build": "tsc -b && vite build", "preview": "vite preview --host 0.0.0.0 --port 5173" }, "dependencies": { "d3": "^7.9.0", "react": "^18.3.1", "react-dom": "^18.3.1", "react-router-dom": "^6.26.2" }, "devDependencies": { "@types/d3": "^7.4.3", "@types/react": "^18.3.5", "@types/react-dom": "^18.3.0", "@vitejs/plugin-react": "^4.3.1", "typescript": "^5.5.4", "vite": "^5.4.2" } } ``` [216 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]