Project Info

LingoSnap

This project did not submit a demo video on Devpost.

Inspiration

There was a story about a Grandpa who learnt how to speak English by placing Post-it notes on everything he could get his hands on, just to converse with his grandsons and grand-daughters that lived far away. This let him actively remember the words because he was always looking at the words. With the invention of the Spectacle, we figured we could utilize this method and make this accessible to everyone so that everyone can actively learn while going through their daily life.

What it does

Using the power of the Spectacles, we created an application that uses a ML model that can detect common objects and classify it under its name and also a translation in the chosen language. This would work as a flashcard so that users can gamify their learning experience. Furthermore, users can screenshot the object which will save it on their phone so that they can access the word and translation anywhere they want.

How we built it

For the Spectacles, we used Snapchat's propietary Lens Studio to create the interface for our application. By integrating our app with SnapML and the object detection models in its asset library, we were able to create a model that could detect objects and shoot a flashcard on top of it. We also utilized PocketBase and Swift for the application part of our project which integrated quite well with the Spectacles.

Challenges we ran into

Lens Studio recently got a major update to allow for easier development for Spectacle apps but its recency caused there to be limited documentation of what we could achieve. Alongside the fact that it was our first time working with anything similar to Lens Studio, a lot of our time was committed to learning how it worked.

Accomplishments we're proud of

We managed to create an Object Detection Model that successfully tracks the objects around us. Created a cohesive Spectacle application that was able to integrate with mobile devices.

What we learned

This was all of out first times learning how to develop for a tool like the Spectacle. It gave us a new angle to approach any future problems and give us first-hand experience to the workflow of someone developing on AR and ML. We also learnt a lot from conversing with the Snapchat team and their struggles and successes with Lens Studio.

What's next

While the app is cohesive, it is still currently in need of more features. We are planning to add voice recognition so that users will be able to practice their pronunciation as well. Furthermore, it would be nice to gamify the app to allow users to continue learning while also having fun. Also, the final goal would be to have the application fully on spectacle and this includes features such as viewing past images and conversations.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

CIn code
JavaScriptIn code
SwiftIn code
TypeScriptIn code

4 of 4 appear in the indexed code.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

844 KB

Source files

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

Feature verification

3 Verified2 Code-supported4 Claimed only1 Blocked

Flashcard-style word/translation displayVerified
Detected objects work as a flashcard for learning, showing word and translation
Claimed on Devposthigh confidence
- Swift_PocketBase/uikitpager/Component/Flashcard.swift:10 — SwiftUI Flashcard view with flip animation showing frontWord and backWord
- Swift_PocketBase/uikitpager/UI/PageContentViewController.swift:87 — Instantiates Flashcard with frontWord/backWord populated from fetched slide data (word/translation)
PocketBase backend integrationVerified
Utilized PocketBase for the application backend
Claimed on Devposthigh confidence
- Swift_PocketBase/pb_migrations/1729368541_created_images.js:2 — PocketBase migration defining the 'images' collection schema
- Swift_PocketBase/uikitpager/UI/SlidePagerViewController.swift:34 — iOS app calls PocketBase REST API at 127.0.0.1:8090 to fetch records
Swift/UIKit mobile companion appVerified
Used Swift for the application part of the project, integrating with the Spectacles
Claimed on Devposthigh confidence
- Swift_PocketBase/uikitpager/UI/SlidePagerViewController.swift:11 — UIKit-based Swift view controller (UIPageViewController) presenting fetched slide/flashcard data
- Swift_PocketBase/uikitpager/AppDelegate.swift:1 — Standard UIKit iOS app entry point present in the project
Lens Studio AR interface for SpectaclesCode-supported
Built using Snapchat's Lens Studio to create the Spectacles application interface
Claimed on Devpostmedium confidence
- Window Popup.js — Lens Studio-style script (uses script.input SceneObject arrays, script exports) that toggles UI window panels, consistent with a Lens Studio project, but no ML/flashcard/detection logic present
- Window Popup 2.js — Second Lens Studio script toggling window visibility; confirms a Lens Studio interface exists but only for basic UI panel switching
Screenshot object to save word/translation on phoneCode-supported
Users can screenshot the object which saves it on their phone to access the word and translation anywhere
Claimed on Devpostmedium confidence
- Swift_PocketBase/pb_migrations/1729368541_created_images.js:3 — PocketBase 'images' collection stores a file, word, and translation, matching the data model needed to persist a captured object's word/translation, but no capture/screenshot-trigger code was found (Lens Studio scripts only handle window toggling)
- Swift_PocketBase/uikitpager/UI/SlidePagerViewController.swift:33 — iOS app fetches saved image/word/translation records from the PocketBase backend for viewing on the phone
Fully on-Spectacle app with history of past images/conversationsClaimed only
Future goal to have the application fully on the Spectacle including viewing past images and conversations
Claimed on Devposthigh confidence
Gamified learning experienceClaimed only
Flashcards work to gamify the user's learning experience
Claimed on Devposthigh confidence
Object detection ML model on Spectacles (SnapML)Claimed only
Uses a ML model with SnapML/object detection to detect common objects on Snapchat Spectacles
Claimed on Devposthigh confidence
Real-time translation of detected objectsClaimed only
Detected objects are classified and translated into a chosen language
Claimed on Devposthigh confidence
Voice recognition for pronunciation practiceBlocked
Planned future addition of voice recognition to practice pronunciation
Claimed on Devposthigh confidence

An AI agent derived these features from the project’s Devpost page and readme, then searched the code for each one. Verified features are backed by cited code; claimed-only features had no supporting code, which is not by itself proof a feature is missing.