# Project export: AeroForge 3D This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: Solving aerospace’s biggest bottleneck—say goodbye to outdated blueprints. The DoD spends millions on manual CAD work. We automate it with AI, in seconds. - Devpost: https://devpost.com/software/aeroforge-3d - GitHub: https://github.com/ShreyesS/AeroForge3D - Demo: http://127.0.0.1:8000/ - Video: https://www.youtube.com/embed/D0c8z_qUi3k?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Shreyes Sridhara (5 commits) ## Devpost submission (written by the team) ### Inspiration Aerospace manufacturers still rely on outdated 2D blueprints for critical aircraft parts—especially for legacy jets like the F-16 and B-52, which were built in the 1960s and 1970s. No CAD models. No 3D visualizations. No streamlined way to modernize these critical assets. During my DoD fellowship, I watched engineers flip through binders of outdated schematics, frustrated by the time and expertise needed to convert them into 3D models for prototyping and repairs. The DoD, Lockheed Martin, and Boeing spend millions just to convert outdated PDFs into usable 3D models and be mission-ready. I saw an opportunity to fix one of aerospace’s biggest bottlenecks by applying state-of-the-art generative AI and visual computing. What if an AI system could instantly reconstruct a 3D model from old blueprints? What if we could automate an entire workflow that takes weeks in seconds, saving millions of dollars and engineering hours? That’s what I built. AeroForge 3D. ### What it does AeroForge 3D is an AI-powered system that transforms legacy 2D blueprints into photorealistic, engineering-accurate 3D models in seconds. 🔹 Defense & Aerospace-Grade Precision → Our system processes military blueprints and ensures compliance with aerospace manufacturing standards (MIL-STD-31000). 🔹 Seamless AI-Powered 2D-to-3D Conversion → Extracts key technical details from PDFs and reconstructs them into high-fidelity, immersive 3D renderings. 🔹 Rapid Prototyping & Upgrades → Engineers can visualize multiple design variations instantly, leading to faster, more informed decision-making. 🔹 Mission-Readiness at Scale → Instead of weeks of painstaking CAD modeling, our AI does it in seconds—allowing defense contractors to be proactive, not reactive. This is more than just automation. It’s a new way to engineer, visualize, and modernize aerospace technology. ### How we built it I combined OCR, AI-powered text processing, OpenAI’s GPT-4o, Sora, Perplexity AI’s Sonar, and Luma AI’s Dream Machine to create a seamless pipeline from blueprint to 3D model. Here’s how I approached the problem, broken down into steps. 1️⃣ Extracting Data from Legacy Blueprints • Used PyMuPDF for OCR (Optical Character Recognition) to extract dimensions, materials, and annotations from technical PDFs. • Parsed structured text with Perplexity AI’s Sonar to generate engineering-grade structured JSON. 2️⃣ Generating the Perfect 3D Prompt • Used OpenAI’s GPT-4o to analyze extracted blueprint data and generate a military-grade Sora prompt with MIL-STD-31000 compliance. • Made sure that the prompt had geometric accuracy, correct material definitions, and compliance verification before visualization. 3️⃣ Creating a 2D Aerospace Visualization • Passed the processed blueprint data into Sora, generating a 2D visualization of the aircraft part. • Once the 2D image was technically accurate, we proceeded to 3D reconstruction. 4️⃣ Bringing It to Life in 3D with Luma AI • Used Luma AI’s Python SDK and REST API to convert the Sora-generated 2D image into a photorealistic 3D rendering. • Leveraged Luma AI’s Dream Machine (Ray-2 model) to create cinematic, immersive 3D animations. • Applied keyframe-based camera motions to showcase part assembly and integration into real-world aircraft. 5️⃣ Guaranteeing Engineering-Grade Usability • AI cross-references generated models against aerospace manufacturing standards to ensure compliance. • Engineers can export AI-generated models for further refinement in CAD, AR, or VR environments. ### Challenges we ran into 🚨 Parsing Legacy Blueprints – Since aerospace PDFs vary in format, it makes OCR tricky. I fine-tuned AI parsing to extract dimensions, tolerances, and materials with precision. 🚨 Ensuring MIL-STD-31000 Compliance – Unlike CAD, my AI had to validate structural integrity and tolerances. I built custom validation logic to ensure military compliance. 🚨 Balancing Realism & Accuracy – Luma’s AI creates stunning renders, but I optimized prompt engineering to keep dimensions engineering-grade while maintaining cinematic quality. 🚨 Optimizing for Speed– Processing high-res PDFs, images, and videos needed fast, scalable API calls to handle real-world defense workloads efficiently. 🚨 Pivoting from Sora API – Since Sora lacks an API, I initially tried Playwright & direct requests but pivoted to perfecting the Sora prompt for seamless integration with Luma AI. 🚨 Building This Solo – After checking into TreeHacks on Friday, I realized I had all the tools necessary to solve this aerospace and defense problem. Working solo meant extra effort, but I took full advantage of the resources and mentorship available at the hackathon to bring this vision to life. ### Accomplishments we're proud of ✅ Automating a Process That Normally Takes Weeks → What used to take weeks of CAD work and engineering hours now happens in seconds. ✅ Seamless AI Pipeline from 2D to 3D → Successfully integrated OCR, structured data extraction, AI-powered prompt generation, and cinematic 3D rendering. ✅ Military & Commercial Applications → This isn’t just for the DoD. It has massive commercial potential for aerospace manufacturers, airlines, and engineering firms. Also (I just learned) for the oil and refinery industry! This problem is surprisingly common. ✅ Game-Changing Visual AI for Engineering → We’re proving that generative AI isn’t just for art—it’s revolutionizing how critical systems are designed, upgraded, and maintained. ### What we learned 💡 AI can bridge the gap between legacy engineering and modern 3D visualization. 💡 Generative AI can serve mission-critical roles beyond entertainment. Luma AI’s Dream Machine isn’t just generating cool videos—it’s reshaping aerospace workflows. 💡 Aerospace & defense need automation, and they need it fast. Millions are wasted every year on manual blueprint conversion—our AI solution is an immediate cost-saver. 💡 CAD isn’t enough. CAD doesn’t automate compliance verification or enhance real-time visualization—our AI does. ### What's next 🚀 AR & VR Integration → Engineers can inspect, modify, and interact with AI-generated 3D models in real-time. 🚀 Automated Compliance Checks → AI-driven MIL-STD-31000 validation ensures defense-grade accuracy. 🚀 Beyond Defense → Expanding to commercial aerospace for faster prototyping and modernization. 🚀 Interactive & Editable Models → Users can tweak AI-generated designs, test integrity, and refine structures. 🚀 Seamless CAD & Manufacturing Integration → Direct CAD exports and real-time AI-assisted prototyping. 🚀 Revolutionizing Aerospace → AeroForge 3D streamlines engineering from blueprint to battlefield. ## README (from the GitHub repository) Solving aerospace’s biggest bottleneck—say goodbye to outdated blueprints. The DoD spends millions on manual CAD work. We automate it with AI, in seconds. Defense tech + 3D reconstruction. AI-powered vision for aerospace legacy parts. Luma's Dream Machine, OpenAI's Sora, and Perplexity AI's Sonar. ## Detected evidence (automated analysis) Indexed codebase: 10 recognized source files, 20 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - Python (language) — detected in the code - Flask (technology) — claimed on Devpost, not found in the code - JavaScript (language) — claimed on Devpost, not found in the code - OpenAI (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (13 of 13) ``` .DS_Store .gitignore README.md src/aero_forge_app/app.py src/aero_forge_app/data_processing.py src/aero_forge_app/image_generation.py src/aero_forge_app/ocr_extraction.py src/aero_forge_app/prompt_generation.py src/aero_forge_app/static/style.css src/aero_forge_app/templates/index.html src/aero_forge_app/templates/results.html src/aero_forge_app/video_generation.py src/basic_aeroforge.ipynb ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - Update README.md - Create README.md - restapi app, luma dream machine 3d rendering of legacy parts, sonar, gpt-4o - first safe commit; also basic aeroforge with luma, perplexity, and openai - Removed sensitive files from tracking ## Key source files (fetched from GitHub, selected and truncated for size) ### src/aero_forge_app/app.py ```python from fastapi import FastAPI, File, UploadFile, Request, Form from fastapi.templating import Jinja2Templates from fastapi.staticfiles import StaticFiles import os from ocr_extraction import extract_text_from_pdf from data_processing import format_text_with_perplexity from prompt_generation import generate_sora_prompt from image_generation import generate_luma_image from video_generation import generate_luma_video app = FastAPI() # ✅ Mount static files app.mount("/static", StaticFiles(directory="static"), name="static") # ✅ Load HTML templates templates = Jinja2Templates(directory="templates") # ✅ Store uploaded PDFs UPLOAD_DIR = "uploads" os.makedirs(UPLOAD_DIR, exist_ok=True) # ✅ Home Page (Upload Form) @app.get("/") def home(request: Request): return templates.TemplateResponse("index.html", {"request": request}) # ✅ Handle PDF Upload + OCR Extraction @app.post("/upload") async def upload_pdf(request: Request, file: UploadFile = File(...)): file_path = os.path.join(UPLOAD_DIR, file.filename) with open(file_path, "wb") as f: f.write(await file.read()) extracted_text = extract_text_from_pdf(file_path) return templates.TemplateResponse("results.html", { "request": request, "filename": file.filename, "extracted_text": extracted_text, "structured_data": None, "image_url": None, "sora_prompt": None, "luma_video": None }) # ✅ Process Text into Structured JSON @app.post("/process-json") async def process_json(request: Request, filename: str = Form(...)): file_path = os.path.join(UPLOAD_DIR, filename) extracted_text = extract_text_from_pdf(file_path) structured_json = format_text_with_perplexity(extracted_text) return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "structured_data": structured_json, "image_url": None, "sora_prompt": None, "luma_video": None }) # ✅ Generate 2D Blueprint Rendering using Luma @app.post("/generate-image") async def generate_image(request: Request, filename: str = Form(...)): file_path = os.path.join(UPLOAD_DIR, filename) extracted_text = extract_text_from_pdf(file_path) structured_json = format_text_with_perplexity(extracted_text) # Generate a text prompt for the Luma AI 2D rendering prompt = f""" Generate a **high-accuracy 2D technical blueprint** of an **{structured_json['component_name']}** in **MIL-STD-31000 engineering format**. **Technical Requirements:** - **Monochrome CAD-style blueprint**, precise **engineering lines & dimensions**. - Display **top, front, and side views** if applicable. - Include key measurements: {structured_json['dimensions']} - Label material properties: **{structured_json['material']}** - Add standard aerospace blueprint gridlines & annotations. - **No artistic shading or color effects**—pure **technical visualization**. """ image_url = generate_luma_image(prompt) return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "structured_data": structured_json, "image_url": image_url, "sora_prompt": None, "luma_video": None }) # ✅ Generate Sora Prompt @app.post("/generate-sora") async def generate_sora(request: Request, filename: str = Form(...), image_url: str = Form(...)): file_path = os.path.join(UPLOAD_DIR, filename) # Retrieve structured JSON data first structured_json = format_text_with_perplexity(file_path) if not structured_json: return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "error": "Failed to generate structured JSON.", "structured_data": None, "sora_prompt": None, "image_url": image_url, # Keep the 2D rendering visible "luma_video": None }) # Generate the Sora prompt from the structured JSON sora_prompt = generate_sora_prompt(structured_json) return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "structured_data": None, # ✅ JSON no longer shown after Sora prompt "sora_prompt": sora_prompt, # ✅ Ensure this is passed "image_url": image_url, # ✅ Keep the 2D image "luma_video": None }) # ✅ Generate Luma 3D Video @app.post("/generate-luma") async def generate_luma(request: Request, filename: str = Form(...), sora_prompt: str = Form(...)): # Generate Luma Dream Machine video luma_video_url = generate_luma_video(sora_prompt) if not luma_video_url: return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "sora_prompt": sora_prompt, "luma_video": None, # Ensure it's None if generation fails "error": "Luma video generation failed.", }) return templates.TemplateResponse("results.html", { "request": request, "filename": filename, "sora_prompt": sora_prompt, "luma_video": luma_video_url, # ✅ Pass generated video URL }) ``` ### src/aero_forge_app/ocr_extraction.py ```python import fitz # PyMuPDF is great for OCR; based on Tesseract-OCR import os from dotenv import load_dotenv load_dotenv() # we need to load env vars here def extract_text_from_pdf(pdf_path): """Extracts text from a given PDF file""" doc = fitz.open(pdf_path) extracted_text = [page.get_text("text") for page in doc] return "\n".join(extracted_text) if __name__ == "__main__": # Example Usage: pdf_path = "uploads/blueprint_compressor_blade.pdf" text = extract_text_from_pdf(pdf_path) print(text) ``` ### src/aero_forge_app/image_generation.py ```python import requests import time import os from dotenv import load_dotenv load_dotenv() # we need to load env vars here from lumaai import LumaAI LUMA_API_KEY = os.getenv("LUMAAI_API_KEY") # Initialize Luma Client client = LumaAI(auth_token=LUMA_API_KEY) def generate_luma_image(prompt, aspect_ratio="4:3", model="photon-1"): """Generates a high-accuracy 2D technical blueprint using Luma AI.""" try: generation = client.generations.image.create( prompt=prompt, aspect_ratio=aspect_ratio, model=model ) # Poll for completion while True: generation = client.generations.get(id=generation.id) if generation.state == "completed": break elif generation.state == "failed": raise RuntimeError(f"Generation failed: {generation.failure_reason}") print("🖼️ Generating 2D Image... (Please wait)") time.sleep(5) # ✅ Get the final image URL image_url = generation.assets.image print(f"✅ Image Generated: {image_url}") return image_url # ✅ Return the image URL to be displayed on the frontend except Exception as e: print(f"❌ Error: {e}") return None ``` ### src/aero_forge_app/data_processing.py ```python import json import re import requests import os PERPLEXITY_API_KEY = os.getenv("PERPLEXITY_API_KEY") def extract_json_from_response(response_text): """Extracts only the JSON block from Perplexity's response.""" json_match = re.search(r"```json\n(.*?)\n```", response_text, re.DOTALL) if json_match: json_str = json_match.group(1).strip() # Extract JSON text try: return json.loads(json_str) # Convert to dictionary except json.JSONDecodeError as e: print("❌ JSON Parsing Error:", e) return None else: print("❌ No JSON block found in response!") return None PERPLEXITY_API_URL = "https://api.perplexity.ai/chat/completions" def format_text_with_perplexity(extracted_text): """Converts blueprint text into structured JSON.""" prompt = f""" Convert the following aerospace blueprint text into structured JSON format with these keys: - component_name - material - dimensions (length, width, height, tolerance if available) - engineering_notes (as a list) ONLY return JSON. Do NOT include explanations. Text: {extracted_text} """ response = requests.post( PERPLEXITY_API_URL, json={"model": "sonar", "messages": [{"role": "user", "content": prompt}]}, headers={"Authorization": f"Bearer {PERPLEXITY_API_KEY}", "Content-Type": "application/json"} ) if response.status_code == 200: response_json = response.json() content = response_json["choices"][0]["message"]["content"] structured_data = extract_json_from_response(content) # ✅ Use the fixed function return structured_data else: print(f"❌ API Error: {response.status_code}, {response.text}") return None ``` ### src/aero_forge_app/video_generation.py ```python import requests import time import os from lumaai import LumaAI from dotenv import load_dotenv load_dotenv() # we need to load env vars here LUMA_API_KEY = os.getenv("LUMAAI_API_KEY") # Initialize Luma Client client = LumaAI(auth_token=LUMA_API_KEY) def generate_luma_video(prompt, aspect_ratio="16:9", model="ray-2", loop=True): """Generates a cinematic aerospace 3D rendering with Luma AI""" try: generation = client.generations.video.create( prompt=prompt, aspect_ratio=aspect_ratio, model=model, loop=loop ) # Polling for completion while True: generation = client.generations.get(id=generation.id) if generation.state == "completed": break elif generation.state == "failed": raise RuntimeError(f"Generation failed: {generation.failure_reason}") print("🎥 Generating Video... (Please wait)") time.sleep(5) # ✅ Debug the full response print(f"🔍 Debugging: generation.assets = {generation.assets}") # ✅ Correct video URL extraction if hasattr(generation.assets, "video"): # Ensure attribute exists video_url = generation.assets.video else: raise ValueError("No video URL found in response.") print(f"✅ Video Generated: {video_url}") # ✅ Save video to local directory video_filename = f'generated_videos/{generation.id}.mp4' os.makedirs("generated_videos", exist_ok=True) response = requests.get(video_url, stream=True) with open(video_filename, 'wb') as file: file.write(response.content) print(f"📂 Video saved as {video_filename}") return video_url except Exception as e: print(f"❌ Error: {e}") return None # ✅ Example Usage (Only runs when this file is executed directly) if __name__ == "__main__": prompt_text = "Generate a high-fidelity 3D render of a Titanium Alloy Compressor Blade, aerospace-grade." video_file = generate_luma_video(prompt_text) ``` ### src/aero_forge_app/prompt_generation.py ```python import openai import json import os from dotenv import load_dotenv load_dotenv() # we need to load env vars here OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") # ✅ Initialize OpenAI client client = openai.OpenAI(api_key=OPENAI_API_KEY) def generate_sora_prompt(blueprint_json): """Generates a MIL-STD-31000-compliant Sora prompt with precise dimensions, materials, and tolerances""" component_name = blueprint_json.get("component_name", "Aerospace Component") material = blueprint_json.get("material", "Unknown Material") dimensions = blueprint_json.get("dimensions", {}) length = dimensions.get("length", "Unknown") width = dimensions.get("width", "Unknown") height = dimensions.get("height", "Unknown") tolerance = dimensions.get("tolerance", "Unknown") engineering_notes = blueprint_json.get("engineering_notes", []) formatted_notes = "\n".join(f"- {note}" for note in engineering_notes) if engineering_notes else "No specific engineering notes provided." user_prompt = f""" Generate a MIL-STD-31000-compliant 3D engineering visualization of a {component_name}, optimized for defense and aerospace applications. **Rendering Style & Compliance Standards:** - **MIL-STD-31000 TDP format**: Use **orthographic projection** with precise **CAD labeling, metadata tags, and GD&T notations**. - **Blueprint Presentation**: - **Monochrome CAD layout** with **white background, black outlines, and labeled dimensions**. - No shading, cinematic effects, or artistic renderings—strictly **technical accuracy**. **Technical Specifications:** - **Component**: {component_name} - **Material**: {material} - **Dimensions**: - **Length**: {length} - **Width**: {width} - **Height**: {height} - **Tolerance**: {tolerance} **Additional Engineering Notes:** {formatted_notes} **Animation & Camera:** - Center the {component_name} in the frame. - Perform a **360-degree isometric rotation** to display all features. - Include an **exploded view animation** showcasing internal structures. **Strict Constraints:** - No photorealism—strictly **engineering CAD visualization**. - No artistic interpretation—adhere strictly to **Technical Data Package standards**. - No cinematic effects—this should resemble official aerospace documentation. Ensure the visualization maintains accurate proportions and follows MIL-STD-31000 specifications. """ # ✅ Use the OpenAI client correctly response = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are an aerospace engineering expert specializing in MIL-STD-31000-compliant visualizations."}, {"role": "user", "content": user_prompt} ], max_tokens=500 ) return response.choices[0].message.content # ✅ Correct way to access response ``` ### src/aero_forge_app/templates/index.html ```html AeroForge3D - AI Blueprint Visualization

AeroForge3D

Upload your aerospace blueprint PDF to generate a high-fidelity 3D visualization.

``` ### src/aero_forge_app/static/style.css ```css /* Global Reset */ * { margin: 0; padding: 0; box-sizing: border-box; font-family: 'Arial', sans-serif; } /* Page Layout */ body { background-color: #0b0c10; color: #c5c6c7; text-align: center; padding: 40px; } /* Main Container */ .container { width: 80%; max-width: 900px; margin: auto; background: #1f2833; padding: 30px; border-radius: 10px; box-shadow: 0px 4px 10px rgba(0, 0, 0, 0.5); } /* Buttons */ button { background-color: #45a29e; color: white; border: none; padding: 12px 18px; font-size: 16px; border-radius: 5px; cursor: pointer; transition: 0.3s; margin-top: 15px; } button:hover { background-color: #66fcf1; } /* Loading Spinner */ .loading { display: none; margin: 20px auto; width: 50px; height: 50px; border: 5px solid rgba(255, 255, 255, 0.2); border-top-color: #66fcf1; border-radius: 50%; animation: spin 0.8s linear infinite; } @keyframes spin { from { transform: rotate(0deg); } to { transform: rotate(360deg); } } /* Preformatted Text Blocks */ pre { background: #0b0c10; padding: 10px; border-radius: 5px; text-align: left; white-space: pre-wrap; word-wrap: break-word; } /* Video Display */ video { width: 100%; max-width: 600px; border-radius: 8px; margin-top: 10px; } /* Image Display */ img { max-width: 500px; margin-top: 10px; border-radius: 8px; } /* Upload Another File */ a.btn { display: inline-block; text-decoration: none; background-color: #45a29e; color: white; padding: 12px 18px; font-size: 16px; border-radius: 5px; transition: 0.3s; margin-top: 15px; } a.btn:hover { background-color: #66fcf1; } ``` ### src/aero_forge_app/templates/results.html ```html Processing Results

Processing Results

Uploaded File: {{ filename }}

{% if extracted_text %}

Extracted OCR Text:

{{ extracted_text }}
{% endif %} {% if structured_data %}

Structured JSON Data

{{ structured_data | tojson(indent=4) }}
{% if image_url %}

2D Blueprint Rendering

2D Blueprint Image {% else %}
{% endif %} {% else %}
{% endif %} {% if image_url %} {% if sora_prompt %}

Sora Prompt:

{{ sora_prompt }}
{% else %}
{% endif %} {% endif %} {% if sora_prompt %} {% if luma_video %}

Luma Dream Machine 3D Visualization

{% else %}
{% endif %} {% endif %}

Upload Another File
```