# Project export: SoleMate This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: From scan to sole – custom 3D printed orthopedics made cheap and easy. - Devpost: https://devpost.com/software/solemate-9jmhuf - GitHub: https://github.com/TaylorT52/solemate - Video: https://www.youtube.com/embed/4OALS7QstDk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — Taylor Tam (23 commits) ## Devpost submission (written by the team) ### Overview With SoleMate, we can create custom shoe insoles in 2 minutes with an iPhone and a $50 3D printer. The average insole costs $300 to $800. ### Inspiration 76% of adults aged 55–64 use foot orthotics, but traditional solutions are often expensive, require specialist visits, and rely on generic sizing rather than true customization. We saw an opportunity to make custom-fit orthotics accessible to everyone using a device people already own—their smartphone. By leveraging iPhone LiDAR technology, we’re eliminating the need for costly appointments and one-size-fits-all solutions. SoleMate empowers users to create 3D-printed, perfectly tailored insoles with a simple scan—because comfort should be personal, not prescribed. ### What it does SoleMate is a mobile app that leverages LiDAR technology on iPhones to scan a user’s feet in 3D and generate a custom-fitted insole. Using this 3D scan, we create a perfectly contoured, 3D-printable shoe insert tailored to the user’s foot shape. Users can then order their custom inserts or print them at home with a 3D printer. ### How we built it Frontend: Built using Swift for a native iOS app experience. 3D Scanning: Utilized Apple’s LiDAR ARKit for high-accuracy foot scanning. From Scan to Custom Insole 1️⃣ Capture the Foot Scan Uses an iPhone LiDAR sensor to collect a high-resolution 3D point cloud of the foot. 2️⃣ Generate a Digital Foot Mesh Densifies and smooths the scan with Open3D and Alpha Shape Reconstruction to create a structured 3D model. 3️⃣ Invert & Shape the Insole The mesh is flipped and refined using Trimesh to ensure the insole fits correctly inside a shoe. 4️⃣ Precision Refinement Aligns the scan with a segmented foot image using OpenCV, fine-tuning the shape with a distance transform algorithm for an optimized fit. 5️⃣ Smoothing & Final Adjustments Applies Laplacian smoothing for a polished finish, removing rough edges and optimizing thickness. 6️⃣ Export for 3D Printing Saves the custom insole as a .OBJ file, ready for 3D printing. Print it yourself or send it to anyone with a 3D printer. 3D Printing Pipeline: Uses Open3D, Trimesh, and ARKit to generate high-fidelity, custom insoles. Saves the custom insole as a .OBJ file, ready for 3D printing. Print it yourself or send it to anyone with a 3D printer. 3D Printing Pipeline: Uses Open3D, Trimesh, and ARKit to generate high-fidelity, custom insoles. ### Challenges we ran into LiDAR Precision: Capturing highly accurate foot dimensions while maintaining a smooth user experience was tricky. 3D Modeling Complexity: Converting foot scan data into a fully adaptable, printable insole required deep work with 3D processing tools. Integration & Optimization: Balancing performance vs. quality—especially ensuring that our app runs efficiently on various iPhone models—posed some hurdles. ### Accomplishments we're proud of 📱 Successfully built a working prototype that scans feet using LiDAR and generates customizable, printable STL files. 🚀 Created a seamless and intuitive UI for easy scanning and ordering. 👟 Pushed the boundaries of custom footwear accessibility—making orthotics available to anyone with a smartphone. ### What we learned Deepened our understanding of LiDAR and 3D modeling. Improved skills in full-stack development and 3D-printing workflows. Refined our ability to build a hardware-integrated application under time pressure. Gained insights into user experience challenges when designing health-tech solutions. ### What's next 📱 Refining the UI/UX for a more streamlined scanning process. 👨‍⚕️ Partnering with podiatrists to validate and improve the fit and comfort of the insoles. 🏭 Exploring mass production using sustainable 3D-printing materials. 🌎 Expanding beyond shoe inserts—3D modeling for virtually anything SoleMate is just the beginning of a future where custom footwear is accessible to everyone, anytime, anywhere. ## README (from the GitHub repository) # Solemate ### TreeHacks project by @taylor tam, @grace yan - upload a sole scan with lidar - get a custom made insole - 3d print it! ### Steps - Create a .ply (point cloud) scan of your foot in-app - Run the provided script (process.py) on your scan - You're given a custom insole which is printable and in a .obj format - Print it yourself, or send this to anyone with a printer ### Devpost: https://devpost.com/software/solemate-9jmhuf ## Detected evidence (automated analysis) Indexed codebase: 28 recognized source files, 85 KB. - Python (language) — detected in the code - Swift (language) — detected in the code ## Codebase structure (from repository index) ### Files (32 of 32) ``` .gitignore feethacks.xcodeproj/project.pbxproj feethacks/ContentView.swift feethacks/feethacks.entitlements feethacks/feethacksApp.swift feethacks/GCPManagement/CredentialManager.swift feethacks/GCPManagement/GCPAuthManager.swift feethacks/GCPManagement/GCPJWTGenerator.swift feethacks/GCPManagement/GCPUploader.swift feethacks/objects/ARManager.swift feethacks/objects/CreatePLY.swift feethacks/objects/FootMeasurement.swift feethacks/objects/FootSegmentation.swift feethacks/objects/PointCloud.swift feethacks/views/ArView.swift feethacks/views/InstructionsView.swift feethacks/views/Metal/MetalContentView.swift feethacks/views/Metal/MetalEnvironment.swift feethacks/views/Metal/MetalPointCloud.swift feethacks/views/Metal/MetalTextureColorThresholdDepth.swift feethacks/views/Metal/MetalTextureColorZap.swift feethacks/views/Metal/MetalTextureView.swift feethacks/views/Metal/MetalTextureViewColor.swift feethacks/views/Metal/MetalTextureViewDepth.swift feethacks/views/Metal/Shaders/shaders.metal feethacks/views/PLYSceneView.swift feethacks/views/WelcomeView.swift feethacksTests/feethacksTests.swift feethacksUITests/feethacksUITests.swift feethacksUITests/feethacksUITestsLaunchTests.swift processing.py README.md ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - Update README.md - Update README.md - Update README.md - Update README.md - Update README.md - Update README.md - added processing script here - Update README.md - added app name - added logo - procrastinating - updated views - added instructions - some UI stuff - added filtering to just get foot - started adding measurements - Generating unique filenames - added file uploader using GCP service account - resolved package issues - cleaning up ## Key source files (fetched from GitHub, selected and truncated for size) ### processing.py ```python import tempfile import os import open3d as o3d import numpy as np import trimesh from trimesh.smoothing import filter_laplacian import shutil import cv2 # --- Your generate_mesh function using Open3D --- def generate_mesh(file_name): pcd = o3d.io.read_point_cloud(file_name) # Densify the point cloud by voxel downsampling pcd = pcd.voxel_down_sample(voxel_size=0.002) # Smaller voxel = denser cloud # Estimate normals pcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.05, max_nn=50)) # Expand each point using its normal to create a denser cloud new_points = [] for point, normal in zip(np.asarray(pcd.points), np.asarray(pcd.normals)): for i in range(20): # Increase density significantly new_points.append(point + normal * (0.001 * np.random.randn())) all_points = np.vstack((np.asarray(pcd.points), np.array(new_points))) pcd.points = o3d.utility.Vector3dVector(all_points) # Re-estimate normals after densification pcd.estimate_normals(search_param=o3d.geometry.KDTreeSearchParamHybrid(radius=0.1, max_nn=30)) pcd.orient_normals_consistent_tangent_plane(k=50) # Define alpha for Alpha Shape Reconstruction alpha = 0.01 tetra_mesh, pt_map = o3d.geometry.TetraMesh.create_from_point_cloud(pcd) mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_alpha_shape(pcd, alpha, tetra_mesh, pt_map) # Apply Laplacian smoothing to reduce sharp triangles and smooth the mesh mesh = mesh.filter_smooth_laplacian(number_of_iterations=10) mesh.compute_vertex_normals() # Create a temporary file for the OBJ with tempfile.NamedTemporaryFile(suffix=".obj", delete=False) as tmp: obj_path = tmp.name o3d.io.write_triangle_mesh(obj_path, mesh) print(f"Generated mesh saved to temporary file: {obj_path}") return obj_path def invert_mesh(input_obj, output_obj=None): """ Invert a convex mesh by reflecting its vertices about its centroid. If no output_obj is provided, a temporary file is created and its path is returned. This effectively “inverts” the shape: - For a convex sole, it turns the outside into an inside cavity. - Note: This does not create a cavity inside a block but simply flips the geometry. """ # If no output path is provided, create a temporary file. if output_obj is None: with tempfile.NamedTemporaryFile(suffix=".obj", delete=False) as tmp: output_obj = tmp.name # Load the input mesh. mesh = trimesh.load(input_obj) # Compute the centroid of the mesh. center = mesh.centroid # Reflect each vertex about the centroid: # new_vertex = center - (old_vertex - center) = 2*center - old_vertex inverted_vertices = 2 * center - mesh.vertices # Create a new mesh with the inverted vertices and same faces. inverted_mesh = trimesh.Trimesh(vertices=inverted_vertices, faces=mesh.faces) # Optionally, invert the face winding so normals point inward. inverted_mesh.invert() # Export the inverted mesh. inverted_mesh.export(output_obj) print("Inverted mesh saved to", output_obj) return output_obj def project_point(point, intrinsics, extrinsics=np.eye(4)): # Convert the 3D point to homogeneous coordinates p_homog = np.append(point, 1) # Transform into camera coordinates (if extrinsics are provided) p_cam = extrinsics @ p_homog x, y, z, _ = p_cam if z == 0: return None # Apply pinhole camera model: u = fx*(x/z) + cx, v = fy*(y/z) + cy u = intrinsics[0, 0] * (x / z) + intrinsics[0, 2] v = intrinsics[1, 1] * (y / z) + intrinsics[1, 2] return np.array([u, v]), z def refine_mesh_with_image(mesh, mask, intrinsics, extrinsics=np.eye(4), step_size=0.001, iterations=10): h, w = mask.shape # Pre-compute a distance transform from the inverted mask: pixels outside the foot have high distance. dist_transform = cv2.distanceTransform(255 - mask, cv2.DIST_L2, 5) vertices = mesh.vertices.copy() for iter in range(iterations): new_vertices = vertices.copy() for i, vertex in enumerate(vertices): proj, depth = project_point(vertex, intrinsics, extrinsics) if proj is None: continue u, v = proj u_int, v_int = int(round(u)), int(round(v)) # Skip if projection is outside the image bounds if u_int < 0 or u_int >= w or v_int < 0 or v_int >= h: continue # Use the distance transform to decide if the point is outside the silhouette. distance = dist_transform[v_int, u_int] # If the distance is above a threshold, adjust the vertex along the ray direction if distance > 1.0: # threshold (adjust as needed) # Compute a direction vector from the camera center through the vertex. p_cam = extrinsics @ np.append(vertex, 1) direction = p_cam[:3] / np.linalg.norm(p_cam[:3]) # Move the vertex a small step along the negative of this direction (bringing it closer) new_vertices[i] = vertex - step_size * direction vertices = new_vertices # update vertices for the next iteration # Update mesh with refined vertices mesh.vertices = vertices return mesh def final_and_smooth(input_obj): mesh = trimesh.load(input_obj) # Load and segment your foot image (here we assume a pre-segmented mask) mask = cv2.imread("/Users/taylortam/Desktop/samples/IMG_0001.jpg", cv2.IMREAD_GRAYSCALE) if mask is None: raise ValueError("Could not load segmentation mask.") # Define your camera intrinsics (example values – adjust as needed) fx, fy = 600, 600 cx, cy = mask.shape[1] / 2, mask.shape[0] / 2 intrinsics = np.array([[fx, 0, cx], [ 0, fy, cy], [ 0, 0, 1]]) [truncated — 1042 more characters] ``` ### feethacks/feethacksApp.swift ```swift // // feethacksApp.swift // feethacks // // Created by Taylor Tam on 2/14/25. // import SwiftUI struct UIViewWrapper: UIViewRepresentable { let view: UIView func makeUIView(context: Context) -> some UIView { view } func updateUIView(_ uiView: UIViewType, context: Context) { } } @main struct feethacksApp: App { //create the ar manager @StateObject var arManager = ARManager() var body: some Scene { WindowGroup { ContentView(arManager: arManager) } } } ``` ### feethacks/ContentView.swift ```swift // // ContentView.swift // feethacks // // Created by Taylor Tam on 2/14/25. // import SwiftUI enum AppMode { case welcome case ar case instructionsView case plyDisplay } struct ContentView: View { @State private var appMode = AppMode.welcome @ObservedObject var arManager: ARManager var body: some View { switch(appMode){ case .welcome: WelcomeView(appMode: $appMode) case .instructionsView: InstructionsView(appMode: $appMode) case .ar: arView(arManager: arManager, appmode: $appMode) case .plyDisplay: PLYSceneView(arManager: arManager, mode: $appMode) } } } ``` ### feethacksUITests/feethacksUITestsLaunchTests.swift ```swift // // feethacksUITestsLaunchTests.swift // feethacksUITests // // Created by Taylor Tam on 2/14/25. // import XCTest final class feethacksUITestsLaunchTests: XCTestCase { override class var runsForEachTargetApplicationUIConfiguration: Bool { true } override func setUpWithError() throws { continueAfterFailure = false } func testLaunch() throws { let app = XCUIApplication() app.launch() // Insert steps here to perform after app launch but before taking a screenshot, // such as logging into a test account or navigating somewhere in the app let attachment = XCTAttachment(screenshot: app.screenshot()) attachment.name = "Launch Screen" attachment.lifetime = .keepAlways add(attachment) } } ``` ### feethacksTests/feethacksTests.swift ```swift // // feethacksTests.swift // feethacksTests // // Created by Taylor Tam on 2/14/25. // import XCTest final class feethacksTests: XCTestCase { override func setUpWithError() throws { // Put setup code here. This method is called before the invocation of each test method in the class. } override func tearDownWithError() throws { // Put teardown code here. This method is called after the invocation of each test method in the class. } func testExample() throws { // This is an example of a functional test case. // Use XCTAssert and related functions to verify your tests produce the correct results. // Any test you write for XCTest can be annotated as throws and async. // Mark your test throws to produce an unexpected failure when your test encounters an uncaught error. // Mark your test async to allow awaiting for asynchronous code to complete. Check the results with assertions afterwards. } func testPerformanceExample() throws { // This is an example of a performance test case. measure { // Put the code you want to measure the time of here. } } } ``` ### feethacksUITests/feethacksUITests.swift ```swift // // feethacksUITests.swift // feethacksUITests // // Created by Taylor Tam on 2/14/25. // import XCTest final class feethacksUITests: XCTestCase { override func setUpWithError() throws { // Put setup code here. This method is called before the invocation of each test method in the class. // In UI tests it is usually best to stop immediately when a failure occurs. continueAfterFailure = false // In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this. } override func tearDownWithError() throws { // Put teardown code here. This method is called after the invocation of each test method in the class. } func testExample() throws { // UI tests must launch the application that they test. let app = XCUIApplication() app.launch() // Use XCTAssert and related functions to verify your tests produce the correct results. } func testLaunchPerformance() throws { if #available(macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 7.0, *) { // This measures how long it takes to launch your application. measure(metrics: [XCTApplicationLaunchMetric()]) { XCUIApplication().launch() } } } } ``` ### feethacks/GCPManagement/CredentialManager.swift ```swift // // CredentialManager.swift // feethacks // // Created by Taylor Tam on 2/15/25. // import Foundation struct ServiceAccountCredentials: Codable { let type: String let project_id: String let private_key_id: String let private_key: String let client_email: String let client_id: String } class CredentialManager { static let shared = CredentialManager() private(set) var credentials: ServiceAccountCredentials? private init() { loadCredentials() } private func loadCredentials() { guard let url = Bundle.main.url(forResource: "service-account", withExtension: "json") else { print("Service account file not found") return } do { let data = try Data(contentsOf: url) let decoder = JSONDecoder() credentials = try decoder.decode(ServiceAccountCredentials.self, from: data) } catch { print("Error loading service account credentials: \(error.localizedDescription)") } } } ``` ### feethacks/views/ArView.swift ```swift // // ArView.swift // feethacks // // Created by Taylor Tam on 2/15/25. // import Foundation import SwiftUI struct arView: View { @ObservedObject var arManager: ARManager @Binding var appmode: AppMode var body: some View { ZStack(alignment: .bottom) { UIViewWrapper(view: arManager.sceneView).ignoresSafeArea() HStack(spacing: 30) { Button { arManager.isCapturing.toggle() } label: { Image(systemName: arManager.isCapturing ? "stop.circle.fill" : "play.circle.fill") } ShareLink(item: PLYFile(pointCloud: arManager.pointCloud), preview: SharePreview("exported.ply")) { Image(systemName: "square.and.arrow.up.circle.fill") } Button() { appmode = AppMode.plyDisplay } label: { Image(systemName: "display") } }.foregroundStyle(.black, .white) .font(.system(size: 50)) .padding(25) } } } ``` ### feethacks/GCPManagement/GCPJWTGenerator.swift ```swift // // GCPJWTGenerator.swift // feethacks // // Created by Taylor Tam on 2/15/25. // import Foundation import SwiftJWT struct GoogleServiceAccountClaims: Claims { let iss: String let scope: String let aud: String let exp: Date let iat: Date } class GCPJWTGenerator { private var credentials: ServiceAccountCredentials? { return CredentialManager.shared.credentials } func generateJWT() throws -> String { guard let creds = credentials else { throw NSError(domain: "GCPJWTGenerator", code: -1, userInfo: [NSLocalizedDescriptionKey: "Credentials not loaded"]) } let iat = Date() let exp = iat.addingTimeInterval(3600) let claims = GoogleServiceAccountClaims( iss: creds.client_email, scope: "https://www.googleapis.com/auth/cloud-platform", aud: "https://oauth2.googleapis.com/token", exp: exp, iat: iat ) var jwt = JWT(claims: claims) let formattedPrivateKey = creds.private_key.replacingOccurrences(of: "\\n", with: "\n") guard let privateKeyData = formattedPrivateKey.data(using: .utf8) else { throw NSError(domain: "GCPJWTGenerator", code: -2, userInfo: [NSLocalizedDescriptionKey: "Invalid private key format"]) } let jwtSigner = JWTSigner.rs256(privateKey: privateKeyData) return try jwt.sign(using: jwtSigner) } } ``` ### feethacks/views/WelcomeView.swift ```swift // // WelcomeView.swift // feethacks // // Created by Taylor Tam on 2/15/25. // import Foundation import SwiftUI struct WelcomeView: View { @Binding var appMode: AppMode @Environment(\.colorScheme) var colorScheme var body: some View { VStack { Spacer() Text("Solemate") .font(.system(size: 35, weight: .bold)) .padding(.vertical) Text("Find your perfect shoe match") .padding(.vertical) Image("heart_foot") // Replace with your asset name .resizable() // Makes the image scalable .scaledToFit() // Ensures it maintains aspect ratio .frame(width: 300, height: 300) // Set desired size Button { appMode = .instructionsView } label: { Text("Create an AR Scan") .font(.headline) .foregroundColor(colorScheme == .dark ? .white : .black) .padding() .frame(width: 200) .background(colorScheme == .dark ? Color.black : Color.white) .cornerRadius(15) .overlay( RoundedRectangle(cornerRadius: 15) .stroke(colorScheme == .dark ? Color.white : Color.black, lineWidth: 2) ) } Spacer() } } } ``` [17 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]