# Project export: ClassX This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: No more boring lectures thanks to Vision Pro + AI video - Devpost: https://devpost.com/software/classx-etk0pl - GitHub: https://github.com/aheze/ClassX - Video: https://www.youtube.com/embed/sDBUnMgTQcU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Andrew Zheng (32 commits), Markus Zhang (7 commits) ## Devpost submission (written by the team) ### Overview Fuse online + offline classes to ‘ultralearn’ (aka cram) class quickly! Best-in-class multi-query RAG (retrieval augmented generation) shows relevant TA notes, LaTeX equations, YouTube videos, and 3b1b videos. Whisper audio transcript can rewind to older transcript entires [Generative Video] Generate 3blue1brown AI videos with GPT-4 and Manim rendering engine!! Exploring the intersection of Mixed Reality, Spatial Computing, and Large Language Models in education. The integration of Vision Pro and AI technologies offers a unique opportunity to enhance learning experiences through immersive, interactive content. Our project, ClassX, is designed to leverage these advancements, focusing on the practical application within an educational context. Our mission with ClassX was to develop a VisionOS app that enriches learning by offering a multi-dimensional platform for students to engage with educational materials. Recognizing the potential for broad application across various fields, we chose to concentrate on education, aiming to provide a solution that addresses the needs of modern learners. ClassX is a virtual learning environment where students can interact with an array of educational resources, including videos, PDFs, and LaTeX-rendered documents, all presented within a virtual lecture hall setting. Features of ClassX Dynamic Educational Material Display: ClassX allows users to navigate through and interact with a variety of media types, all integrated seamlessly into the user’s visual field. AI-Driven Content Personalization: We use TogetherAI and Mistral Mixture of Experts to customize the learning experience. We adapt the lecturer's style into a form that fits individual learning styles, preferences, and progress. Engaging Learning Methods: Through interactive elements, such as quizzes and exercises, ClassX transforms traditional learning materials into engaging, interactive experiences. Comprehensive Academic Integration: ClassX offers direct access to a wide range of academic resources, employing sophisticated search technologies to provide relevant, up-to-date materials. Development Process The development of ClassX was characterized by a collaborative and methodical approach, emphasizing efficiency and technical innovation. The team utilized a range of development tools and platforms, including Xcode, OpenAI API, Mistral, and LaTeX libraries, to create a robust educational platform. Stack Server Together AI: Mistral 7x8b Mixture of Experts chat model OpenAI: text-embedding-ada-3 embedding Chroma multi query vector search. Each document and transcript maps to many keys, and Chromadb reranks n->n SQL mapping by similarity. 3b1b Manim: Grant Sanderson’s Python math rendering engine. GPT-4 generates 2d animation scenes as executable code, creating 10-sec crystal-clear AI animated video (without OpenAI Sora 😉). FastAPI: serves generated video and APIs visionOS App 100% Swift and SwiftUI: fully native app! handles animations, images, webviews, and more native visionOS dynamic layout grids and resizing support without breakpoints handles animations, images, webviews, and more native visionOS dynamic layout grids and resizing support without breakpoints Whisper (Local): Transcribe audio offline with timestamps Live streaming via AVFoundation Live streaming via AVFoundation LaTeX renderer (with regex to extract LaTeX sections and handle inlining) The ClassX Experience Virtual Lecture Halls: ClassX adds a virtual environment on top of an existing one. It enhances/supplements the boring lecture hall experience with educational content to facilitate a comprehensive and engaging learning experience. Customized Learning Journeys: AI technology assesses each learner's unique profile to deliver personalized content, optimizing the educational experience. Interactive Learning Tools: ClassX enhances learning retention through interactive quizzes and exercises, providing instant feedback to reinforce understanding. Challenges and Achievements Developing for Vision Pro: Tailoring ClassX to the innovative capabilities of Vision Pro required creative problem-solving and technical acumen. Complex Content Integration: The integration of LaTeX into mixed reality posed significant challenges, but the team successfully achieved smooth rendering of intricate academic content. Future Directions Moving forward, ClassX aims to expand its content offerings, integrate live tutoring capabilities, and explore the potential of augmented reality (AR) for practical learning applications. Considerations While ClassX represents a significant step forward in educational technology, we are conscious of the challenges ahead, including device accessibility and data privacy concerns. Thank you for exploring ClassX. Our work represents a commitment to advancing educational technology for a brighter, more informed future. ## README (from the GitHub repository) # 👓 ClassX AI Classrooms in Vision Pro + GPT-4 Generative 3b1b Videos ![ClassX Interface](./pi_transparent.png) [![ClassX Transcript](./ClassXGif.gif)](https://www.youtube.com/watch?v=sDBUnMgTQcU) _Submitted to TreeHacks 2024, Andrew Zheng_ - **Integrate Learning Environments**: ClassX synergizes online and offline learning methodologies, enabling students to rapidly assimilate class material through advanced 'ultralearning' techniques. - **Advanced Resource Access Gateway (RAG)**: Utilizes a sophisticated multi-query RAG system to display pertinent teaching assistant notes, LaTeX-rendered equations, educational YouTube content, and 3blue1brown (3b1b) animations, directly within the student's field of vision. - **Dynamic Audio Transcription**: Incorporates Whisper audio transcription technology, offering the unique capability to **_rewind_** through lecture transcripts and revisit previous search results, enhancing comprehension and retention. - **✨ Generative Video Creation ✨**: Employs GPT-4's advanced generative capabilities, coupled with the Manim rendering engine, to produce educational videos in the style of 3blue1brown, offering custom, high-quality visual explanations of complex concepts. # Tech Stack Server - **Together AI:** Mistral 7x8b Mixture of Experts chat model - **OpenAI:** text-embedding-ada-3 embedding - **Chroma** multi query vector search. Each document and transcript maps to many keys, and Chromadb reranks n->n SQL mapping by similarity. - **3b1b Manim**: Grant Sanderson’s Python math rendering engine. GPT-4 generates 2d animation scenes as executable code, creating 10-sec crystal-clear AI animated video (without OpenAI Sora 😉). visionOS - **FastAPI**: serves generated video and APIs visionOS App - **100% Swift and SwiftUI**: fully native app! - handles animations, images, webviews, and more - native visionOS dynamic layout grids and resizing support without breakpoints - **Whisper (Local)**: Transcribe audio offline with timestamps - Live streaming via AVFoundation - **LaTeX renderer** (with regex to extract LaTeX sections and handle inlining) ## Setup Good luck 🫠 ... But seriously, kindly DM us on [Twitter](https://x.com/photon_mz) if you really want help. But first you pay an entry fee of $3500 for the Vision Pro LMFAO ## Detected evidence (automated analysis) Indexed codebase: 22 recognized source files, 82 KB. - Python (language) — detected in the code - Swift (language) — detected in the code - FastAPI (technology) — claimed on Devpost, not found in the code - JavaScript (language) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (50 of 50) ``` README.md server/.editorconfig server/.gitignore server/gen_manim.py server/main.py server/query.py server/scrape_3b1b.py server/scrape_mit.py server/service.py WhisperTest.xcodeproj/project.pbxproj WhisperTest.xcodeproj/project.xcworkspace/contents.xcworkspacedata WhisperTest.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist WhisperTest.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved WhisperTest.xcodeproj/project.xcworkspace/xcuserdata/aheze.xcuserdatad/IDEFindNavigatorScopes.plist WhisperTest.xcodeproj/xcuserdata/aheze.xcuserdatad/xcdebugger/Breakpoints_v2.xcbkptlist WhisperTest.xcodeproj/xcuserdata/aheze.xcuserdatad/xcschemes/xcschememanagement.plist WhisperTest/Assets.xcassets/AccentColor.colorset/Contents.json WhisperTest/Assets.xcassets/BottomRightArrow.imageset/Contents.json WhisperTest/Assets.xcassets/Contents.json WhisperTest/Assets.xcassets/HorizontalArrow.imageset/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Back.solidimagestacklayer/Content.imageset/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Back.solidimagestacklayer/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Front.solidimagestacklayer/Content.imageset/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Front.solidimagestacklayer/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Middle.solidimagestacklayer/Content.imageset/Contents.json WhisperTest/Assets.xcassets/VisionAppIcon.solidimagestack/Middle.solidimagestacklayer/Contents.json WhisperTest/Model.swift WhisperTest/openai_whisper-base.en/AudioEncoder.mlmodelc/metadata.json WhisperTest/openai_whisper-base.en/AudioEncoder.mlmodelc/model.mil WhisperTest/openai_whisper-base.en/config.json WhisperTest/openai_whisper-base.en/generation_config.json WhisperTest/openai_whisper-base.en/MelSpectrogram.mlmodelc/metadata.json WhisperTest/openai_whisper-base.en/MelSpectrogram.mlmodelc/model.mil WhisperTest/openai_whisper-base.en/TextDecoder.mlmodelc/metadata.json WhisperTest/openai_whisper-base.en/TextDecoder.mlmodelc/model.mil WhisperTest/TestingData.swift WhisperTest/Utilities/Utilities.swift WhisperTest/Views/ContentView.swift WhisperTest/Views/TranscriptView.swift WhisperTest/Views/Visualizations/DetailVisualization.swift WhisperTest/Views/Visualizations/ImageVisualization.swift WhisperTest/Views/Visualizations/LatexVisualization.swift WhisperTest/Views/Visualizations/URLVisualization.swift WhisperTest/Views/Visualizations/VisualizationsView.swift WhisperTest/WhisperTestApp.swift WhisperTest/WhisperViewModel.swift WhisperTest/WhisperVM+Listen.swift WhisperTest/WhisperVM+Testing.swift WhisperTest/WhisperVM+Visualizations.swift ``` ### Dependencies No dependency index available. ### Recent commits (newest first) - Update README.md - cleaner readme headline - readme - add pi - add pi - video telemetry - bug - rewind media - Merge remote-tracking branch 'refs/remotes/origin/main' - Remove code - gen 3b1b videos lfg - Add new window for latex - Fix background thread - Test on actual vision pro - Base - Working basic visualizations - Add base blur - Work on transcript - Work on transcript - Working resize ## Key source files (fetched from GitHub, selected and truncated for size) ### server/main.py ```python from service import * from fastapi import FastAPI from fastapi.responses import FileResponse import os import time app = FastAPI() @app.post("/search") async def search(data: WhisperUpload): transcript = "\n".join(data.segments) completion = Together.chat(Prompts.transcript_to_bullets(transcript, n=4)) bullet_points = list(filter(None, completion.replace('```', '').split('\n- '))) print(bullet_points) queries = OpenAI.embed(bullet_points) k = Chroma.query(query_embeddings=queries, n_results=3) k = {key: flatten_2d(v) for key, v in k.items() if key and v} results = [SearchResult(id=a, distance=b, metadata=c) for a,b,c in zip(k['ids'], k['distances'], k['metadatas'])] results.sort(key=lambda x: x.distance) unique_results = SearchResult.set(results, by='id') topk = [x.metadata for x in unique_results[:20]] # take top 3 scores. could be unique, or same lecture diff bullet. # urls = list(set([x['path'] for x in topk]))[:3] # take urls of best 3b1b and mit results # mit unlabelled source by default. dirty. first_3b1b = next((x for x in topk if x.get('source') == '3b1b'), None) first_mit = next((x for x in topk if not x.get('source')), None) urls = [x['path'] for x in [first_3b1b, first_mit] if x] bullets = "\n".join(["- " + x['bullet'] for x in topk]) print(bullets, urls) gen_recitation = Together.chat(Prompts.bullets_to_recitation(bullets), max_tokens=1000) print(gen_recitation) visuals = [ *[Visualization( id=url, visualizationType="url", mainBody=url) for url in urls], Visualization( id=f"recitation_{int(time.time())}", visualizationType="latex", mainBody=gen_recitation), # Visualization( # id=f"bullets_{int(time.time())}", visualizationType="bullet", mainBody=bullets) ] return ServerResponse(visualizations=visuals, id=data.id, uploadNumber=data.uploadNumber) @app.post('/video') async def video(data: WhisperUpload): transcript = "\n".join(data.segments) slug = Manim.gen(transcript) # media type is hosted on this content server. url is base/slug visual = Visualization(id=f"video-{int(time.time())}", visualizationType="media", mainBody=slug) return ServerResponse(visualizations=[visual], id=data.id, uploadNumber=data.uploadNumber) @app.get("/file/{name_file:path}") async def get_file(name_file: str): return FileResponse(path=os.getcwd() + "/" + name_file) @app.get('/') async def ping(): return "I am a teapot" if __name__ == "__main__": import uvicorn uvicorn.run("main:app", host="0.0.0.0", port=3000, reload=True) ``` ### WhisperTest/WhisperVM+Visualizations.swift ```swift // // WhisperVM+Visualizations.swift // WhisperTest // // Created by Andrew Zheng (github.com/aheze) on 2/18/24. // Copyright © 2024 Andrew Zheng. All rights reserved. // import SwiftUI extension WhisperViewModel { func displayVisualizations(visualizations: [Visualization]) { self.displayedVisualizations = visualizations } } ``` ### WhisperTest/WhisperTestApp.swift ```swift // // WhisperTestApp.swift // WhisperTest // // Created by Andrew Zheng (github.com/aheze) on 2/17/24. // Copyright © 2024 Andrew Zheng. All rights reserved. // import SwiftUI @main struct WhisperTestApp: App { var body: some Scene { WindowGroup { ContentView() } .windowStyle(.plain) WindowGroup(for: Visualization.self) { $visualization in if let visualization { VisualizationDetail(visualization: visualization) } } .defaultSize(width: 900, height: 700) } } ``` ### server/gen_manim.py ```python from manim import * class GeneratedGIF(Scene): def construct(self): # Setup the matrix matrix_m = Matrix([[0.5, 0.5], [0.4, 0.6]], v_buff=1.0, h_buff=1.0).scale(0.7) # Header text header = Text("Squaring Markov Matrix", color=BLUE).to_edge(UP) # Square the matrix descriptor squared_matrix_description = Text( "Squaring the matrix keeps its properties", font_size=24 ).next_to(matrix_m, DOWN) # Animation: Show the header, then the matrix, and finally the description self.play(Create(header)) self.play(Write(matrix_m), run_time=1) self.play(Write(squared_matrix_description), run_time=1) ``` ### server/scrape_mit.py ```python from glob import glob from pypdf import PdfReader import json import re from service import * from tqdm import tqdm pairs = [] for i,f in enumerate(tqdm(glob('content/18.06/static_resources/*sum.pdf'))): reader = PdfReader(f) url = f"https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011/{f.split('/')[-1]}" number_of_pages = len(reader.pages) pages = "\n\n".join([page.extract_text() for page in reader.pages]) prompt = Prompts.transcript_to_bullets(pages) completion = Together.chat(prompt) # read as kinda-yaml ignoring ':' bullet_points = list(filter(None, completion.replace('```', '').split('\n- '))) pair = MultiKey(keys=bullet_points, value=pages, embeds=OpenAI.embed(bullet_points), path=url, unit=grep_unit(f)) Chroma.add(**pair.chroma) pairs += [pair] query = OpenAI.embed('Markov matrices') k = Chroma.query(query_embeddings=query, n_results=5) print(k['metadatas']) with open('content/18.06_summaries.json', 'w') as f: json.dump(pairs, f, cls=DataclassJSONEncoder) ``` ### WhisperTest/Model.swift ```swift // // Model.swift // WhisperTest // // Created by Andrew Zheng (github.com/aheze) on 2/17/24. // Copyright © 2024 Andrew Zheng. All rights reserved. // import SwiftUI struct WhisperUpload: Codable { var id: String = UUID().uuidString // every time I upload, this is incremented var uploadNumber: Int // all segments. Each upload, focus on the newest segments. Is there enough change to warrant a new set of visualizations? var segments: [String] } enum VisualizationType: String, Codable { case latex case url case image case plainText case bullet } struct Visualization: Codable, Identifiable, Hashable { // a unique identifier. No two visualizations should have the same ID. var id: String var visualizationType: VisualizationType // for latex: the latex // for url: the url // for image: the url of the image // for plainText: just the text to show var mainBody: String? // e.g. 3b1b // var sourceTitle: String? } struct ServerResponse: Codable { // should match the ID of the WhisperUpload var id: String var uploadNumber: Int var visualizations: [Visualization] } ``` ### server/scrape_3b1b.py ```python import json import re from service import * from tqdm import tqdm from youtube_transcript_api import YouTubeTranscriptApi # yt video ids for 3b1b Linear Algebra series ids = [ "fNk_zzaMoSs", "k7RM-ot2NWY", "kYB8IZa5AuE", "XkY2DOUCWMU", "rHLEWRxRGiM", "Ip3X9LOh2dk", "uQhTuRlWMxw", "v8VSDg_WQlA", "LyGKycYT2v0", "eu6i7WJeinw", "BaM7OCEm3G0", "jBsC34PxzoM", "P2LTAUO1TdA", "PFDu9oVAE-g", "e50Bj7jn9IQ", "TgKwz5Ikpc8" ] yt_transcripts = list(map(YouTubeTranscriptApi.get_transcript, tqdm(ids))) pairs = [] for i,(id, t) in enumerate(tqdm(list(zip(ids, yt_transcripts)))): url = f"https://www.youtube.com/embed/{ids[i]}" subtitles = Subtitles(t) prompt = Prompts.transcript_to_bullets(subtitles.text) completion = Together.chat(prompt) # read as kinda-yaml ignoring ':' bullet_points = list(filter(None, completion.replace('```', '').split('\n- '))) print(bullet_points) pair = MultiKey(keys=bullet_points, value=subtitles.text, embeds=OpenAI.embed(bullet_points), path=url, unit=id, source='3b1b') Chroma.add(**pair.chroma) pairs += [pair] query = OpenAI.embed('What is Cramer\'s rule?') k = Chroma.query(query_embeddings=query, n_results=5, where={"source": "3b1b"}) print(k['metadatas']) with open('content/3b1b_linealg_summaries.json', 'w') as f: json.dump(pairs, f, cls=DataclassJSONEncoder) ``` ### server/query.py ```python from service import * # transcript = """ # What's a Markov matrix? # Can I just write down a typical Markov matrix, say .1, .2, .7, .01, .99 0, let's say, .3, .3, .4. Okay. There's a -- a # totally just invented Markov matrix. What makes it a Markov matrix? # Two properties that this -- this matrix has. # So two properties are -- one, every entry is greater equal zero. # All entries greater than or equal to zero. # And, of course, when I square the matrix, the entries will still be greater/equal zero. # I'm going to be interested in the powers of this matrix. # """ transcript = """ With determinants it's a fascinating, small topic inside linear algebra. Used to be determinants were the big thing, and linear algebra was the little thing, but they -- those changed, that situation changed. Now determinants is one specific part, very neat little part. And my goal today is to find a formula for the determinant. It'll be a messy formula. So that's why you didn't see it right away. But if I'm given this n by n matrix then I use those entries to create this number, the determinant. So there's a formula for it. """ completion = Together.chat(Prompts.transcript_to_bullets(transcript, n=4)) bullet_points = list(filter(None, completion.replace('```', '').split('\n- '))) print(bullet_points) queries = OpenAI.embed(bullet_points) k = Chroma.query(query_embeddings=queries, n_results=3) k = {key: flatten_2d(v) for key, v in k.items() if key and v} results = [SearchResult(id=a, distance=b, metadata=c) for a,b,c in zip(k['ids'], k['distances'], k['metadatas'])] results.sort(key=lambda x: x.distance) unique_results = SearchResult.set(results, by='id') print(unique_results) topk = [x.metadata for x in unique_results[:20]] # take top 3 scores. could be unique, or same lecture diff bullet. urls = list(set([x['path'] for x in topk])) bullets = "\n".join(["- " + x['bullet'] for x in topk]) print(bullets, urls) gen_recitation = Together.chat(Prompts.bullets_to_recitation(bullets), max_tokens=1000) print(gen_recitation) ``` ### WhisperTest/WhisperVM+Testing.swift ```swift // // WhisperVM+Testing.swift // WhisperTest // // Created by Andrew Zheng (github.com/aheze) on 2/18/24. // Copyright © 2024 Andrew Zheng. All rights reserved. // import SwiftUI import WhisperKit extension WhisperViewModel { func startTestingScript() { let chunkLength = Float(2.0) let wordLength = Float(0.19) guard let testingConfiguration else { return } let chunked = testingConfiguration.mockTranscript.components(separatedBy: .newlines) Task { var previousChunk: String? func addPreviousChunk(currentIndex: Int) { let previousIndex = currentIndex - 1 if let previousChunk { let segment = TranscriptionSegment( // id: index, id: UUID().uuidString, seek: 0, start: Float(previousIndex) * chunkLength, end: Float(currentIndex) * chunkLength, text: previousChunk, tokens: [], temperature: 1, avgLogprob: 1, compressionRatio: 1, noSpeechProb: 1 ) confirmedSegments.append(segment) if testingConfiguration.useSnapshotsForVisualizations { print("using index: \(previousIndex)") if testingConfiguration.snapshots.indices.contains(previousIndex) { let visualizations = testingConfiguration.snapshots[previousIndex].visualizations serverResponseBySegmentID[segment.id] = ServerResponse(id: segment.id, uploadNumber: 0, visualizations: visualizations) displayVisualizations(visualizations: visualizations) } else { print("No visualizations??") displayVisualizations(visualizations: []) } } } } for index in chunked.indices { let chunk = chunked[index] await { @MainActor in if previousChunk != nil { addPreviousChunk(currentIndex: index) } self.currentText = "" }() let words = chunk.components(separatedBy: .whitespaces) for wordIndex in words.indices { await { @MainActor in self.currentText += "\(words[wordIndex]) " }() try await Task.sleep(for: .seconds(Double(wordLength))) } previousChunk = chunk try await Task.sleep(for: .seconds(0.3)) } // last one await { @MainActor in self.currentText = "" addPreviousChunk(currentIndex: chunked.count) }() } } } ``` ### WhisperTest/WhisperVM+Listen.swift ```swift // // WhisperVM+Listen.swift // WhisperTest // // Created by Andrew Zheng (github.com/aheze) on 2/18/24. // Copyright © 2024 Andrew Zheng. All rights reserved. // import SwiftUI extension WhisperViewModel { func listen() { // upload reactToSegmentsChange() } func reactToSegmentsChange() { $confirmedSegments .dropFirst() .throttle(for: .seconds(8), scheduler: RunLoop.main, latest: true) .sink { [weak self] confirmedSegments in guard let self else { return } if let testingConfiguration = self.testingConfiguration, testingConfiguration.useSnapshotsForVisualizations { return } let confirmedSegments = confirmedSegments.suffix(10) guard let last = confirmedSegments.last else { return } let segments = confirmedSegments.map { $0.text.trimmingCharacters(in: .whitespacesAndNewlines) } let upload = WhisperUpload(id: last.id, uploadNumber: self.currentUploadNumber, segments: segments) let encoder = JSONEncoder() encoder.outputFormatting = .prettyPrinted do { let data = try encoder.encode(upload) self.upload(data: data) guard let string = String(data: data, encoding: .utf8) else { print("Couldn't make string from data") return } self.currentUploadNumber += 1 print("Received data!!!!") print(string) } catch { print("Error encoding: \(error)") } } .store(in: &cancellables) } func upload(data: Data) { // Define the URL object if let url = URL(string: "https://7eb4-68-65-169-179.ngrok-free.app/search") { var request = URLRequest(url: url) request.httpMethod = "POST" request.addValue("application/json", forHTTPHeaderField: "Content-Type") request.httpBody = data // Create a URLSession data task let task = URLSession.shared.dataTask(with: request) { data, response, error in guard let data = data, error == nil else { print("Error: \(error?.localizedDescription ?? "No error description")") return } // Handle the response here if let httpStatus = response as? HTTPURLResponse, httpStatus.statusCode != 200 { // Check for http errors print("StatusCode should be 200, but is \(httpStatus.statusCode)") print("Response = \(response!)") } // Try to parse the JSON data do { // if let jsonResponse = try JSONSerialization.jsonObject(with: data, options: []) as? [String: Any] { // print("Response JSON: \(jsonResponse)") // } let decoder = JSONDecoder() let serverResponse = try decoder.decode(ServerResponse.self, from: data) DispatchQueue.main.async { self.serverResponseBySegmentID[serverResponse.id] = serverResponse if let currentFocusedSegmentID = self.currentFocusedSegmentID { } else { let isLatest = self.confirmedSegments.last?.id == serverResponse.id print("isLatest? \(isLatest)") if isLatest { self.displayVisualizations(visualizations: serverResponse.visualizations) } } } print("serverResponse: \(serverResponse)") } catch { print("Failed to parse JSON:", error) } } // Start the task task.resume() } } } ``` [11 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]