Project Info

Winner

Chroma: Best AI application using Chroma

Artki.tech

Devpost

Inspiration

The spark came during a late-night furniture shopping session. We watched a friend spend hours on Pinterest creating mood boards, then struggle to find matching furniture on multiple websites. Despite having great taste, they couldn't translate their vision into reality. We realized this was a universal problem: the gap between inspiration and implementation in interior design. Nearly 98% of design enthusiasts are dissatisfied with their home decor, yet over $25 billion is spent on interior design each year in the U.S. While AI had revolutionized many creative fields, interior design remained fragmented across inspiration platforms, furniture retailers, and 3D planning tools. Our breakthrough insight: what if we could learn someone's aesthetic preferences the same way Spotify learns musical taste - not through questionnaires, but through choices? Museum artworks became our "songs" - universally recognized pieces that reveal deep aesthetic preferences through simple A/B comparisons. What It Does Artki.tech is an AI-powered interior design platform that creates personalized 3D room visualizations with real, purchasable furniture based on your unique taste profile. Here's the complete flow: Floorplan Analysis: Users upload their room's floorplan, which gets transformed into a 3D room reconstruction using Claude-generated Three.js code. Floorplan Analysis: Users upload their room's floorplan, which gets transformed into a 3D room reconstruction using Claude-generated Three.js code. Taste Discovery: Users compare 12 pairs of museum artworks (Monet vs. Picasso, Van Gogh vs. Warhol). Each choice updates a 512-dimensional taste vector using CLIP embeddings, learning aesthetic preferences without questionnaires. Taste Discovery: Users compare 12 pairs of museum artworks (Monet vs. Picasso, Van Gogh vs. Warhol). Each choice updates a 512-dimensional taste vector using CLIP embeddings, learning aesthetic preferences without questionnaires. Real Product Discovery: Our system scrapes Amazon to find actual furniture products matching your taste profile, ensuring everything you see can be purchased. Real Product Discovery: Our system scrapes Amazon to find actual furniture products matching your taste profile, ensuring everything you see can be purchased. AI-Powered 3D Pipeline: Amazon products are enhanced with OpenAI-generated images Images are converted to 3D models using Claude's Three.js generation Models are rendered in your reconstructed room AI-Powered 3D Pipeline: Amazon products are enhanced with OpenAI-generated images Images are converted to 3D models using Claude's Three.js generation Models are rendered in your reconstructed room Interactive Visualization: Real-time Three.js rendering shows actual purchasable furniture in your specific room layout. Users can rotate, zoom, and rearrange items to perfect their design. Interactive Visualization: Real-time Three.js rendering shows actual purchasable furniture in your specific room layout. Users can rotate, zoom, and rearrange items to perfect their design. Smart Recommendations: ChromaDB's vector search finds semantically similar products based on visual features, style tags, and your evolving preferences. Smart Recommendations: ChromaDB's vector search finds semantically similar products based on visual features, style tags, and your evolving preferences. Architecture Overview Frontend: Next.js 16 + React 19 + TypeScript + Three.js + React Three Fiber Backend: FastAPI + Python 3.11 AI/ML: Claude 4.5 Sonnet + CLIP embeddings + OpenAI API + NeRF Database: ChromaDB Cloud for vectors 3D: Three.js + React Three Fiber + NeRF reconstruction Data Source: Amazon product scraping for real furniture Development Process & Technical Evolution Initial Approach: NeRF Neural Radiance Fields We started by training NeRF (Neural Radiance Fields) models to create photorealistic 3D reconstructions of rooms from just a few photos Built a Flask API server (api_server.py) to handle NeRF training with user-uploaded images Successfully generated 3D room reconstructions from 10+ photos per room Pivot Decision: While NeRF worked, the output had low fidelity and training took 8-10 minutes per room Day 1: Foundation & Pivot Set up Next.js frontend and FastAPI backend Integrated Three.js for 3D visualization Key Pivot: Switched from NeRF to Claude API for faster, higher-quality 3D generation Claude generates Three.js code directly, which we render with React Three Fiber Reduced generation time from 30 minutes to 3 seconds Day 2: AI Integration & Product Pipeline Implemented CLIP embeddings for artwork analysis Built taste vector algorithm for preference learning Amazon Integration: Created web scraping system to find real products matching user preferences Image-to-3D Pipeline: Scrape Amazon for furniture matching user taste Use OpenAI API to generate product images Convert images to 3D models using Claude Render models in Three.js scene Scrape Amazon for furniture matching user taste Use OpenAI API to generate product images Convert images to 3D models using Claude Render models in Three.js scene Day 3: Complete System Integration Connected floorplan upload to room reconstruction Integrated user preference embeddings with product search Built complete pipeline: Floorplan → Room Reconstruction → Preference Analysis → Amazon Scraping → Image Generation → 3D Conversion → Final Visualization Polished UI/UX with TailwindCSS Complete Technical Pipeline Room Reconstruction: User uploads floorplan → Claude generates Three.js room geometry from floorplan specifications Room Reconstruction: User uploads floorplan → Claude generates Three.js room geometry from floorplan specifications Taste Learning: CLIP encodes museum artworks into 512-dimensional vectors. User choices update their vector: user_vec = user_vec + win_vec - 0.5 * lose_vec Taste Learning: CLIP encodes museum artworks into 512-dimensional vectors. User choices update their vector: user_vec = user_vec + win_vec - 0.5 * lose_vec Product Discovery: Scrape Amazon for furniture matching taste vector Extract product metadata (price, dimensions, materials) Generate embeddings for semantic search Product Discovery: Scrape Amazon for furniture matching taste vector Extract product metadata (price, dimensions, materials) Generate embeddings for semantic search 3D Model Generation: OpenAI API transforms product descriptions into detailed images Images fed to Claude for Three.js code generation Real-time rendering of actual products as 3D models 3D Model Generation: OpenAI API transforms product descriptions into detailed images Images fed to Claude for Three.js code generation Real-time rendering of actual products as 3D models Final Visualization: Combine room reconstruction with generated furniture models in interactive 3D scene Final Visualization: Combine room reconstruction with generated furniture models in interactive 3D scene Challenges We Ran Into 1. NeRF Training Performance Problem: NeRF neural radiance field training took 15-30 minutes per room with low-fidelity output. Solution: Pivoted to Claude API for instant Three.js code generation, reducing time from 30 minutes to 3 seconds while improving quality. 2. Amazon Product Integration Problem: Connecting abstract taste preferences to real, purchasable products. Solution: Built web scraper for Amazon products, then used embeddings to match products to user taste vectors. 3. Image-to-3D Conversion Pipeline Problem: No direct way to convert Amazon product images to 3D models. Solution: Created multi-step pipeline: Amazon data → OpenAI image enhancement → Claude 3D code generation → Three.js rendering. 4. 3D Model Loading Issues Problem: GLTF files referenced external textures, causing 404 errors. Solution: Converted to self-contained GLB format with embedded textures. 5. Embedding Performance Problem: Generating CLIP embeddings for scraped products took 30+ seconds. Solution: Pre-computed embeddings stored in ChromaDB, reducing search to <100ms. 6. AI Code Generation Reliability Problem: Claude sometimes generated invalid Three.js code for complex furniture. Solution: Implemented robust error handling with fallback hand-crafted generators for common furniture types. 7. Floorplan to 3D Reconstruction Problem: Converting 2D floorplan images to accurate 3D room geometry. Solution: Used Claude to interpret floorplan and generate proportional Three.js room structures. Accomplishments That We're Proud Of Successfully Pivoted from NeRF: Started with neural radiance fields, recognized limitations, and pivoted to a better solution within 24 hours Successfully Pivoted from NeRF: Started with neural radiance fields, recognized limitations, and pivoted to a better solution within 24 hours End-to-End Pipeline: Built complete pipeline from floorplan upload to 3D room with real Amazon products - a truly functional prototype End-to-End Pipeline: Built complete pipeline from floorplan upload to 3D room with real Amazon products - a truly functional prototype Multi-AI Orchestration: Successfully integrated 5 different AI systems (Claude, CLIP, OpenAI, ChromaDB, NeRF) into one seamless experience Multi-AI Orchestration: Successfully integrated 5 different AI systems (Claude, CLIP, OpenAI, ChromaDB, NeRF) into one seamless experience Real Product Integration: Connected abstract preferences to actual purchasable Amazon products, solving a real-world problem Real Product Integration: Connected abstract preferences to actual purchasable Amazon products, solving a real-world problem Real-time 3D Generation: Reduced 3D generation from 30 minutes (NeRF) to 3 seconds (Claude) while improving quality Real-time 3D Generation: Reduced 3D generation from 30 minutes (NeRF) to 3 seconds (Claude) while improving quality Lightning-Fast Search: <100ms semantic search across scraped products with 512-dimensional vectors Lightning-Fast Search: <100ms semantic search across scraped products with 512-dimensional vectors Production-Ready Architecture: Built scalable, well-documented codebase that could be deployed tomorrow Production-Ready Architecture: Built scalable, well-documented codebase that could be deployed tomorrow What We Learned Technical Insights Multi-modal AI is powerful: Combining visual (CLIP) and language (Claude) AI creates emergent capabilities Vector databases are game-changers: ChromaDB enabled instant personalized search Prompt engineering matters: Structured prompts improved Claude's code generation by 70% Fallbacks are essential: Every AI component needs a reliable backup Product Insights Users want simplicity: 12 comparisons hit the sweet spot between accuracy and user patience Visual choices reveal preferences: People make faster, more confident decisions with images than questionnaires 3D visualization sells: Seeing furniture in context dramatically improves user confidence Team Insights API integration complexity: Coordinating multiple external APIs requires careful error handling Performance optimization is iterative: Each bottleneck revealed led to architectural improvements What's Next for Artki.tech Immediate Goals (Next Month) Real Furniture Integration: Partner with IKEA, Wayfair, West Elm for actual purchasable items Mobile AR Preview: Use ARKit/ARCore for in-room visualization Expand Catalog: 500+ furniture items with real product links Business Model Freemium: Free taste profiling + 3 room designs Pro Subscription: Unlimited designs, high-res exports, AR preview Affiliate Revenue: Commission from furniture purchases Enterprise API: White-label solution for furniture retailers

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only
  • AnthropicIn code
  • CSSIn code
  • FastAPIIn code
  • HTMLIn code
  • Hugging FaceIn code
  • JavaScriptIn code
  • Next.jsIn code
  • OpenAIIn code
  • PythonIn code
  • PyTorchIn code
  • ReactIn code
  • Tailwind CSSIn code
  • TypeScriptIn code
  • FlaskClaimed
  • VercelClaimed

13 of 15 appear in the indexed code. 2 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

459 KB

Source files

88

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

0 stars