# Project export: Project InSight This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 12.0 - Tagline: A hardware-agnostic Swift-based OS/Frontend for augmented reality devices. - Devpost: https://devpost.com/software/project-insight-048pqk - GitHub: https://github.com/Aephn/calhacks-Insight - Video: https://www.youtube.com/embed/4LF6GEb4Z_I?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Aephn (58 commits), AndrewPreston (8 commits) ## Devpost submission (written by the team) ### Overview (Apologies for the hard-to-see display of the widgets, its hard to take a video of a screen in glasses.) GIT REPO CAN BE FOUND HERE: https://github.com/Aephn/calhacks-Insight ### Inspiration We all like augmented reality hardware, but find the user experience to be lacking, the designs unappealing, and the utility to be near non-existent. We decided to create a platform to address these criticisms of the augmented reality landscape. ### What it does Project InSight is a pair of understated augmented reality glasses that are designed to operate without an internal computer and interface, either wired or wireless, with a tethered device. This allows us to achieve better battery life, comfort, energy efficiency, and at a lower cost. Alongside the head mounted display, the InSight software allows for an unmatched degree of user customization with a near infinite number of information displayable, the separation between your minds eye and reality is non-existent. ### How we built it We first began spec'ing out the hardware we were going to use to demo the software. We went with a pair of Sony μ-OLED displays originally designated for automotive applications with the intent of using them for stereoscopic 3D, a series of custom ribbon cables and a small driver board to allow them to be interfaced with via Type-C Display Port. That display port connection is either used to connect internally to a SBC performing a P2P connection with an external computer or as a direct connection to a pocket computer such as a smartphone. After settling on the hardware we spent the vast majority of the hackathon chipping away at the software. ### Challenges we ran into We designed a custom frame that we were going to 3D print, but because of the internet issues plaguing Calhacks we were unable to do so. As a result, we were unable to create a dual-screen implementation of our test hardware that had a satisfactory user experience because of display alignment issues. The lack of custom frames was the root cause of numerous packaging issues that ultimately lead us to abandon our wireless prototype in favor of a lighter, tethered approach. ### Accomplishments we're proud of We were able to produce both custom hardware and software in a time crunch despite numerous roadblocks. From facing design challenges on what our roadmap should look like to understanding what is optimal for our hardware packaging, we were able to leap through each hurdle with a creative fix. ### What we learned We learned more about the development of creating an app through Swift, along with putting together hardware to match with our operating system. We also learned from experimenting with multiple different types of integrations and working amongst group members to create an equal and quick workflow to reach our deadline. ### What's next We plan to involve more integrations with other APIs to provide even more widgets and create a more compact packaging in the future. We would also look into a wireless solution along with range extender options for an even more seamless experience for the user. Another feature we have on our roadmap is to offer dual display lenses to create stereoscopic 3D views for the user to get an even more immersive view. ## README (from the GitHub repository) # Insight (CalHacks 25') ![alt text](InSight/Assets/ReadMe/s1.jpeg) This repo is the IOS backend in Swift handling the displaying of widgets and other associated pieces of software for the associated AR/VR glasses (hardware agnostic) for the Insight project, created for the CalHacks 2025' hackathon. ## Inspiration We all like augmented reality hardware, but find the user experience to be lacking, the designs unappealing, and the utility to be near non-existent. We decided to create a platform to address these criticisms of the augmented reality landscape. Many "smart glasses" overly emphasize AI-first features, while neglecting the fundementals. Thus, we decided to implement software ourselves. ## What it does Project InSight is a pair of understated augmented reality glasses that are designed to operate without an internal computer and interface, either wired or wireless, with a tethered device. This allows us to achieve better battery life, comfort, energy efficiency, and at a lower cost. Alongside the head mounted display, the InSight software allows for an unmatched degree of user customization with a near infinite number of information displayable, the separation between your minds eye and reality is non-existent. ## How we built it ![alt text](InSight/Assets/ReadMe/glasses.jpg) We first began spec'ing out the hardware we were going to use to demo the software. We went with a pair of Sony μ-OLED displays originally designated for automotive applications, a series of custom ribbon cables and a small driver board to allow them to be interfaced with via Type-C Display Port. That display port connection is either used to connect internally to a SBC performing a P2P connection with an external computer or as a direct connection to a pocket computer such as a smartphone. After settling on the hardware we spent the vast majority of the hackathon chipping away at the software. ## Challenges we ran into We designed a custom frame that we were going to 3D print, but because of the internet issues plaguing Calhacks we were unable to do so. As a result, we were unable to create a dual-screen implementation of our test hardware that had a satisfactory user experience because of display alignment issues. The lack of custom frames was the root cause of numerous packaging issues that ultimately lead us to abandon our wireless prototype in favor of a lighter, tethered approach. ## Accomplishments that we're proud of We were able to produce both custom hardware and software in less than two days, despite numerous roadblocks. ## What we learned Nobody expects the Spanish Inquisition. ## What's next for Project InSight The world may never know... ## Images ![alt text](InSight/Assets/ReadMe/s1.jpeg) ![alt text](InSight/Assets/ReadMe/s2.png) ![alt text](InSight/Assets/ReadMe/s3.jpeg) ![alt text](InSight/Assets/ReadMe/s4.jpeg) ![alt text](InSight/Assets/ReadMe/s5.jpeg) ![alt text](InSight/Assets/ReadMe/wid1.png) ![alt text](InSight/Assets/ReadMe/banner1.png) ![alt text](InSight/Assets/ReadMe/logo.jpeg) ![alt text](InSight/Assets/ReadMe/aiai.png) ## Detected evidence (automated analysis) Indexed codebase: 25 recognized source files, 211 KB. - Swift (language) — detected in the code ## Codebase structure (from repository index) ### Files (42 of 42) ``` .gitignore InSight.xcodeproj/project.pbxproj InSight.xcodeproj/project.xcworkspace/contents.xcworkspacedata InSight.xcodeproj/xcshareddata/xcschemes/InSight.xcscheme InSight.xcodeproj/xcuserdata/harrisonlin.xcuserdatad/xcschemes/xcschememanagement.plist InSight/Assets.xcassets/AccentColor.colorset/Contents.json InSight/Assets.xcassets/AppIcon.appiconset/Contents.json InSight/Assets.xcassets/Contents.json InSight/Assets.xcassets/happy.imageset/Contents.json InSight/Assets.xcassets/hearts.imageset/Contents.json InSight/Assets.xcassets/mkt-1.imageset/Contents.json InSight/Assets.xcassets/ModelY.imageset/Contents.json InSight/Assets.xcassets/pensive.imageset/Contents.json InSight/Assets.xcassets/quirky.imageset/Contents.json InSight/Assets.xcassets/talk.imageset/Contents.json InSight/ContentView.swift InSight/Info.plist InSight/InSight.entitlements InSight/InSightApp.swift InSight/Item.swift InSight/Models/Space.swift InSight/Models/WidgetModels.swift InSight/Overlays/IncomingCallOverlay.swift InSight/Overlays/ShopOverlay.swift InSight/Overlays/SpacesOverlay.swift InSight/Overlays/WidgetEditOverlay.swift InSight/Overlays/WidgetMenuOverlay.swift InSight/Widgets/AiAiWidget.swift InSight/Widgets/BatteryWidget.swift InSight/Widgets/CalendarWidget.swift InSight/Widgets/CarWidget.swift InSight/Widgets/ClockWidget.swift InSight/Widgets/HealthRingsWidget.swift InSight/Widgets/MinimapWidget.swift InSight/Widgets/NowPlayingWidget.swift InSight/Widgets/StockWidget.swift InSight/Widgets/WeatherWidget.swift InSightTests/InSightTests.swift InSightUITests/InSightUITests.swift InSightUITests/InSightUITestsLaunchTests.swift LICENSE.md README.md ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - fix(readme): error link - Merge pull request #5 from Aephn/aiai-refac - readme(pics): add final photos and modify readme. hopefully this is good - feat(dragging): set draggable widgets only when holding down on the widget. (I love my life - feat(tts): add tts to aiai with uc berkeley jokes cause yknow 3am coding makes me wanna crash out - feat(aiai redo): implement backup prompts for aiai if api fails. - feat(spaces): implement a spaces feature for storing multiple presets. - feat(player): impl nowplaying functionality. - fix(nowplaying): fix lost nowplaying rework. - feat(shop): add shop overlay. - feat(aiai widget) fix aiai formatting, add scaling to the widget. - feat(aiai): incorporate aiai into new ui. - Merge pull request #4 from Aephn/battery - fix(widgets): rearrange for better formatting. - feat(AiAi): Added a little Dood with Claude, he no work, but he be there. - feat(stocks): add a fully functioning stock module that uses the AlphaVantage API to display realtime stock data with a corresponding graph. Must add an API key in .env. - feat(battery): add new battery widget, and remove unecessary text. Battery widget needs to be tested on a real device, however. - chore(widget): change widget name from glass effect to liquid glass - feat(buttons): Implement holding down the screen to open the menu to add widgets, simulate calls, etc. - feat(market): re-add market button but on homepage. ## Key source files (fetched from GitHub, selected and truncated for size) ### LICENSE.md ```markdown Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control s [truncated — 8357 more characters] ``` ### InSight/Item.swift ```swift // // Item.swift // InSight // // Created by Harrison Lin on 10/24/25. // import Foundation import SwiftData @Model final class Item { var timestamp: Date init(timestamp: Date) { self.timestamp = timestamp } } ``` ### InSightTests/InSightTests.swift ```swift // // InSightTests.swift // InSightTests // // Created by Harrison Lin on 10/24/25. // import Testing @testable import InSight struct InSightTests { @Test func example() async throws { // Write your test here and use APIs like `#expect(...)` to check expected conditions. } } ``` ### InSightUITests/InSightUITestsLaunchTests.swift ```swift // // InSightUITestsLaunchTests.swift // InSightUITests // // Created by Harrison Lin on 10/24/25. // import XCTest final class InSightUITestsLaunchTests: XCTestCase { override class var runsForEachTargetApplicationUIConfiguration: Bool { true } override func setUpWithError() throws { continueAfterFailure = false } @MainActor func testLaunch() throws { let app = XCUIApplication() app.launch() // Insert steps here to perform after app launch but before taking a screenshot, // such as logging into a test account or navigating somewhere in the app let attachment = XCTAttachment(screenshot: app.screenshot()) attachment.name = "Launch Screen" attachment.lifetime = .keepAlways add(attachment) } } ``` ### InSight/InSightApp.swift ```swift // // InSightApp.swift // InSight // // Created by Harrison Lin on 10/24/25. // import SwiftUI import SwiftData @main struct InSightApp: App { @UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate var sharedModelContainer: ModelContainer = { let schema = Schema([ Item.self, ]) let modelConfiguration = ModelConfiguration(schema: schema, isStoredInMemoryOnly: false) do { return try ModelContainer(for: schema, configurations: [modelConfiguration]) } catch { fatalError("Could not create ModelContainer: \(error)") } }() var body: some Scene { WindowGroup { ContentView() } .modelContainer(sharedModelContainer) } } class AppDelegate: NSObject, UIApplicationDelegate { func application(_ application: UIApplication, supportedInterfaceOrientationsFor window: UIWindow?) -> UIInterfaceOrientationMask { return [.landscapeLeft, .landscapeRight] } } ``` ### InSightUITests/InSightUITests.swift ```swift // // InSightUITests.swift // InSightUITests // // Created by Harrison Lin on 10/24/25. // import XCTest final class InSightUITests: XCTestCase { override func setUpWithError() throws { // Put setup code here. This method is called before the invocation of each test method in the class. // In UI tests it is usually best to stop immediately when a failure occurs. continueAfterFailure = false // In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this. } override func tearDownWithError() throws { // Put teardown code here. This method is called after the invocation of each test method in the class. } @MainActor func testExample() throws { // UI tests must launch the application that they test. let app = XCUIApplication() app.launch() // Use XCTAssert and related functions to verify your tests produce the correct results. } @MainActor func testLaunchPerformance() throws { // This measures how long it takes to launch your application. measure(metrics: [XCTApplicationLaunchMetric()]) { XCUIApplication().launch() } } } ``` ### InSight/Models/WidgetModels.swift ```swift // // WidgetModels.swift // InSight // // Created by Harrison Lin on 10/25/25. // import SwiftUI // widget struct struct WidgetCard: Identifiable, Codable { let id: UUID var position: CGPoint // Changed from gridPosition to absolute position var size: CGSize // Changed to CGSize for pixel-based sizing var type: WidgetType var parameters: WidgetParameters? // Optional parameters for customization } // Widget parameters for customization struct WidgetParameters: Codable { var textColor: CodableColor? var backgroundColor: CodableColor? var borderColor: CodableColor? var fontSize: CGFloat? var useGlassEffect: Bool? // Enable liquid glass effect var temperatureUnit: TemperatureUnit? // Fahrenheit or Celsius var scale: CGFloat? // Scale for widgets (minimap, health rings, etc.) var useDynamicBatteryColor: Bool? // Enable dynamic battery color feature var stockSymbol1: String? // First stock symbol var stockSymbol2: String? // Second stock symbol var enableTTS: Bool? // Enable Text-to-Speech for AiAi widget } // Helper to make Color codable struct CodableColor: Codable, Equatable { var red: Double var green: Double var blue: Double var opacity: Double init(color: Color, opacity: Double = 1.0) { // Extract RGB values from UIColor let uiColor = UIColor(color) var r: CGFloat = 0, g: CGFloat = 0, b: CGFloat = 0, a: CGFloat = 0 uiColor.getRed(&r, green: &g, blue: &b, alpha: &a) self.red = Double(r) self.green = Double(g) self.blue = Double(b) self.opacity = Double(a) } var color: Color { Color(red: red, green: green, blue: blue, opacity: opacity) } // Custom equality that accounts for floating point precision static func == (lhs: CodableColor, rhs: CodableColor) -> Bool { abs(lhs.red - rhs.red) < 0.01 && abs(lhs.green - rhs.green) < 0.01 && abs(lhs.blue - rhs.blue) < 0.01 && abs(lhs.opacity - rhs.opacity) < 0.01 } } // type of widget // ADD HERE IF ADDING WIDGETS!! enum WidgetType: String, Codable, CaseIterable { case clock case nowPlaying case healthRings case weather case minimap case calendar case car case battery case stock case placeholder case incomingCall case aiAi } // Health data structure struct ActivityRingsData { var moveRing: Double = 0.0 var exerciseRing: Double = 0.0 var standRing: Double = 0.0 } // Weather data structure struct WeatherData { var temperature: Double = 72.0 // Fahrenheit var condition: WeatherCondition = .sunny } enum WeatherCondition: String, Codable { case sunny case cloudy case raining case thunderstorming } enum TemperatureUnit: String, Codable { case fahrenheit case celsius } // Preference key for tracking widget sizes struct WidgetSizePreferenceKey: PreferenceKey { static var defaultValue: CGSize = .zero static func reduce(value: inout CGSize, nextValue: () -> CGSize) { value = nextValue() } } ``` ### InSight/Overlays/ShopOverlay.swift ```swift // // ShopOverlay.swift // InSight // // Created by Harrison Lin on 10/26/25. // import SwiftUI struct ShopOverlay: View { @Binding var showShop: Bool var body: some View { ZStack { // Background overlay Color.black.opacity(0.8) .ignoresSafeArea() .onTapGesture { withAnimation(.easeInOut(duration: 0.3)) { showShop = false } } // Main shop content VStack(spacing: 0) { // Header HStack { Text("Widget Shop") .font(.system(size: 28, weight: .bold)) .foregroundColor(.white) Spacer() // Close button Button(action: { withAnimation(.easeInOut(duration: 0.3)) { showShop = false } }) { Image(systemName: "xmark") .font(.system(size: 16, weight: .medium)) .foregroundColor(.white.opacity(0.7)) .frame(width: 32, height: 32) .background(Color.white.opacity(0.1)) .clipShape(Circle()) } } .padding(.horizontal, 24) .padding(.top, 24) .padding(.bottom, 16) // Market image display ScrollView { VStack(spacing: 20) { // Display the market image Image("mkt-1") .resizable() .aspectRatio(contentMode: .fit) .cornerRadius(12) .shadow(color: Color.black.opacity(0.3), radius: 10, x: 0, y: 5) // Optional: Add description text // Text("Simulated Widgets Marketplace") // .font(.system(size: 18, weight: .medium)) // .foregroundColor(.white.opacity(0.8)) // .padding(.top, 8) Text("Explore available widgets for your InSight experience") .font(.system(size: 14)) .foregroundColor(.white.opacity(0.6)) .multilineTextAlignment(.center) .padding(.horizontal, 32) .padding(.bottom, 40) } .padding(20) } .frame(maxHeight: 600) } .frame(maxWidth: 700) .background( RoundedRectangle(cornerRadius: 24) .fill(Color.black.opacity(0.95)) .overlay( RoundedRectangle(cornerRadius: 24) .stroke(Color.white.opacity(0.1), lineWidth: 1) ) ) .shadow(color: Color.black.opacity(0.5), radius: 20, x: 0, y: 10) .padding(.horizontal, 40) } .transition(.opacity.combined(with: .scale(scale: 0.9))) } } #Preview { ShopOverlay(showShop: .constant(true)) .background(Color.gray) } ``` ### InSight/Widgets/ClockWidget.swift ```swift // // ClockWidget.swift // InSight // // Created by Harrison Lin on 10/25/25. // import SwiftUI struct ClockWidget: View { let widget: WidgetCard let currentTime: Date var body: some View { // Extract parameters with defaults let textColor = widget.parameters?.textColor?.color ?? Color.white.opacity(0.85) let bgColor = widget.parameters?.backgroundColor?.color ?? Color.black.opacity(0.95) let borderColor = widget.parameters?.borderColor?.color ?? Color.black.opacity(0.98) let fontSize = widget.parameters?.fontSize ?? 32 let useGlass = widget.parameters?.useGlassEffect ?? false VStack(spacing: 4) { Text(currentTime, style: .time) .font(.system(size: fontSize, weight: .bold, design: .rounded)) .foregroundColor(textColor) .minimumScaleFactor(0.5) .lineLimit(1) } .padding(.horizontal, 20) .padding(.vertical, 12) .background( ZStack { // Only use glass effect on iOS 26+ and if enabled if #available(iOS 26.0, *), useGlass { // Liquid glass effect - works on black backgrounds RoundedRectangle(cornerRadius: 20) .fill( LinearGradient( gradient: Gradient(colors: [ Color.white.opacity(0.08), Color.white.opacity(0.03), Color.white.opacity(0.05) ]), startPoint: .topLeading, endPoint: .bottomTrailing ) ) .overlay( // Inner glow RoundedRectangle(cornerRadius: 20) .stroke( LinearGradient( gradient: Gradient(colors: [ Color.white.opacity(0.4), Color.white.opacity(0.1), Color.clear ]), startPoint: .topLeading, endPoint: .bottomTrailing ), lineWidth: 2 ) ) .overlay( // Shimmer/highlight effect RoundedRectangle(cornerRadius: 20) .fill( LinearGradient( gradient: Gradient(stops: [ .init(color: Color.white.opacity(0.15), location: 0), .init(color: Color.clear, location: 0.3) ]), startPoint: .topLeading, endPoint: .center ) ) .padding(1) ) .shadow(color: Color.white.opacity(0.1), radius: 20, x: 0, y: 0) .shadow(color: Color.black.opacity(0.5), radius: 10, x: 0, y: 5) } else { // Standard solid dark background for iOS < 26 or when glass is disabled RoundedRectangle(cornerRadius: 20) .fill(bgColor) .overlay( RoundedRectangle(cornerRadius: 20) .stroke(borderColor, lineWidth: 1) ) } } ) } } ``` ### InSight/Overlays/IncomingCallOverlay.swift ```swift // // IncomingCallOverlay.swift // InSight // // Created by Harrison Lin on 10/25/25. // import SwiftUI struct IncomingCallOverlay: View { let callerName: String @Binding var isReceivingCall: Bool var body: some View { VStack { // Compact call notification banner at the top HStack(spacing: 10) { // Green gradient phone icon on the left ZStack { RoundedRectangle(cornerRadius: 8) .fill( LinearGradient( gradient: Gradient(colors: [ Color(white: 0.20), Color(white: 0.15), Color(white: 0.1), Color(white: 0.05) ]), startPoint: .top, endPoint: .bottom ) ) .frame(width: 36, height: 36) Image(systemName: "phone.fill") .font(.system(size: 24)) .foregroundStyle( LinearGradient( gradient: Gradient(colors: [Color.green, Color.green.opacity(0.5)]), startPoint: .topLeading, endPoint: .bottomTrailing ) ) } // "Call Incoming: Caller Name" text Text("Call Incoming: \(callerName)") .font(.system(size: 15, weight: .semibold)) .foregroundColor(.white) } .padding(.horizontal, 20) .padding(.vertical, 6) .background( ZStack { // Base dark layer RoundedRectangle(cornerRadius: 28) .fill(Color(white: 0.1).opacity(0.8)) // Linear gradient fill RoundedRectangle(cornerRadius: 28) .fill( LinearGradient( gradient: Gradient(colors: [ Color.white.opacity(0.1), Color.white.opacity(0.02), Color.white.opacity(0.1) ]), startPoint: UnitPoint(x: 1, y: 1), endPoint: UnitPoint(x: 0, y: 0) ) ) .blur(radius: 5) // Radial gradient shimmer border RoundedRectangle(cornerRadius: 28) .stroke( RadialGradient( gradient: Gradient(stops: [ .init(color: Color.white.opacity(0.15), location: 0.0), .init(color: Color.white.opacity(0.05), location: 0.5), .init(color: Color.black.opacity(0.3), location: 1.0) ]), center: .topLeading, startRadius: 10, endRadius: 200 ), lineWidth: 1.5 ) } ) .shadow(color: Color.black.opacity(0.3), radius: 15, x: 0, y: 5) .padding(.top, 16) .onTapGesture { // Tap to dismiss withAnimation(.easeInOut(duration: 0.7)) { isReceivingCall = false } } .onAppear { // Auto-dismiss after 5 seconds DispatchQueue.main.asyncAfter(deadline: .now() + 5.0) { withAnimation(.easeInOut(duration: 0.7)) { isReceivingCall = false } } } Spacer() } .frame(maxWidth: .infinity) } } ``` [14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]