Project Info

Moli

This project did not submit a demo video on Devpost.

Inspiration

Between learning languages in school, or utilizing Duolingo, there is one big pain point - immersion. You learn a lot about the language, but much of it might not be practical for you. That’s why we created Moli, where the world around you becomes your classroom.

What it does

Moli allows you to immerse yourself within your own environment and learn the translations of daily objects and moments of your own life.

How we built it

We combined the snapchat AR spectacles SDK with Vision learning models finetuned on hugginface and marked objects with a label in english and their spanish translation. We also utilized the new snapchat supported supabase integration to save these objects into on our database for a unique learning experience for each and every user.

Challenges we ran into

Figuring out how to use the Snapchat-Spectacles SDK was quite difficult at first but as we kept messing around with the glasses we were able to get a hang of developing on the spectacles!

Accomplishments we're proud of

We are proud of accomplishing depth caching for our objects with an intuitive UX/UI display above each object detailing the translation of each object we see.

What we learned

We learned how to develop AR applications using Lens Studio and had a blast building with Snapchat!

What's next

We would like to iterate on the user experience and optimize the latency between sending an image request to our huggingface backend instance and getting an object label -> translation -> bounding box on the snap spectacles

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only

JavaScriptIn code
TypeScriptIn code
Google GeminiClaimed
Hugging FaceClaimed
SupabaseClaimed

2 of 5 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

10 MB

Source files

1,726

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.