# Project export: LinguaVista - Language & Cultural Learning Platform This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: See, speak, and connect with others around the world through AI-powered, immersive visual learning. - Devpost: https://devpost.com/software/linguavista-language-cultural-learning-platform - GitHub: https://github.com/hcshires/LinguaVista - Video: https://www.youtube.com/embed/SEXIPMBIg-o?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 3 GitHub contributor(s) — Henry Shires (15 commits), hcshires (5 commits), Raina Song (3 commits) ## Devpost submission (written by the team) ### Overview 💡 ### Inspiration 💡 Despite a digitally-centered world that is the most connected in all of human history, many students, communities, and schools across the world still lack access to the internet. And with research as well as people claiming visual learning is beneficial for those learning complex subjects and engaging with culture, limited access to information, especially lots of visual aids and resources on the internet, makes education that relies on verbal communication challenging. The benefits of remote education tools were clearly illustrated as the world faced a global pandemic, forcing schools to push resources online, and further hurting communities that lack digital connection, and as the world continues to further become digital-centric, more solutions are needed to ensure future generations are educated no matter their location on the globe. A US elementary school teacher known by the LinguaVista team expressed her desire for more opportunities in the classroom to show students visuals and experiences for them to learn about rich subjects like science, social studies, and language arts (English as well as other languages). She specifically cited the lack of opportunities (i.e. field trips) for elementary students to receive authentic experiences of the world, the people, and the ideas around them. She also mentioned how hard it is to implement solutions in the classroom due to certain families not being able to participate at home without access to the internet. According to the U.S. Department of Education's Journal of Education and Practice, frameworks such as Problem-Based Learning (PBL) and Higher-Order Thinking (HOT) have demonstrated the need and effectiveness for lessons and curriculum involving visual aids. It claims "learners understand information better in the classroom when they see it...such as images, flowcharts, diagrams, video, simulations, graphs, cartoons, coloring books, slide shows/PowerPoint decks, posters, movies, games, and flash cards (Kohnke, Lucas)." This research urges the use of such resources, of which it may not be possible for educational institutions to implement, either due to technical complexity, limited budgets, or other reasons. And if the research claims visual aids work in the classroom, it can be assumed for individuals wanting to learn subjects individually, information is better obtained visually for them as well. To solve this divide between the need for global education and the many populations without connectivity, a solution requires other forms of technology to bridge the resource gap. With the rise and improvement of generative artificial intelligence performance and accuracy, the idea of placing models on bare metal or on physical devices enables the ability to create a source of knowledge offline from network connectivity, including image generation and two-way conversation between a person and an AI model. Our team was inspired to create LinguaVista to maximize the adoption of digital, visual, and experiential learning tools across the world via the breakthrough technology of Edge AI. LinguaVista ensures no internet connection is required to learn more about our complex world, including ways to enjoy powerful conversations about subject matter. ⚙️ ### What it does ⚙️ LinguaVista connects people with experiences and conversation through an all-in-one Edge AI application. Users select a subject, search for a concept they are curious about—such as a new language, culture, geography, or scientific idea—and connect with one of three options: a real person fluent in the language, an expert available online, or a generative text-to-speech and image AI model hosted on their device, which operates completely offline and independently of external services. Users can have conversations with either type of interaction, either to practice language fluency or learn more about their subject and then conclude them with AI summaries, generated visuals matching the conversation, and saved conversation notes to their profile. 🏗️ ### How we built it 🏗️ 🟣 Edge AI 🟣 LinguaVista boasts the ability to utilize AI all on the local application–no internet necessary–thanks to Edge AI. Our stack includes a Whisper model that transcribes spoken input into text. This text is then processed by a LLaMa3 model, which generates accurate and contextually relevant responses. To enhance user experience, a Coqui model converts these responses back into speech, enabling a fluid and natural conversation. Additionally, Stable Diffusion 3 generates AI-powered visuals that complement discussions, reinforcing learning through visual representation. By combining speech, text, and image generation, LinguaVista ensures a highly engaging and effective AI-driven educational platform, all while maintaining the capability to function offline and independently of external services. 🔴 Zoom 🔴 LinguaVista integrates Zoom's SDK to facilitate seamless video and audio communication, enhancing its AI-driven conversational learning experience. After a user initiates a query, a chat interface powered by Zoom enables real-time, two-way interaction with the AI. The platform leverages Zoom’s RTMS server to process incoming audio and video packets, which is utilized by the AI stack to generate feedback to the user providing an interactive and engaging user experience. ⚫ Education ⚫ LinguaVista’s ability to operate as a standalone desktop application through Electron and React make it an exceptionally accessible and inclusive educational tool, aligning with its mission to provide knowledge and learning opportunities to individuals without reliable internet access. ensures that users can engage in interactive conversations, speech-based learning, and visual-enhanced education without depending on external or cloud-based services. This independence from the internet makes LinguaVista an ideal solution for rural communities, remote learners, and individuals in low-connectivity regions, empowering users with high-quality, AI-driven education anytime, anywhere. 🚩 ### Challenges we ran into As a team with lots of diverse ideas, tech interests, and perspectives, we took a lot of time to thoroughly iterate our idea to include the features we wanted. While this choice matured our idea, it may have led to us losing prototyping time. Within our application, lots of technology is involved because of our complex scope. From multiple AI models processing speech and returning reasoned information, to various APIs communicating data back and forth, not to mention all of these modules are running on the same device, we had lots of difficulty ensuring each piece lined up correctly through integration. Additionally these AI models have to run optimally to ensure performance and accuracy are held to good standard on the user's device Our stack is large, and with a large stack comes LOTS of mix-matched dependencies… Zoom RTMS is not fully prepared as a feature, so we were forced to work around a prototype version which limited the scope of our goals to utilize Zoom API. A fully released version of Zoom RTMS would allow us to easily match like minded users to chat with each other in addition to the option of chatting with our AI. 🥇 ### Accomplishments we're proud of We've had great collaboration with Zoom and Liquid AI sponsors while trying to pull off this idea. From interfacing with Zoom's emerging SDK technology and working with their team on our unique use case, to getting feedback from Liquid on our application of Edge AI technology to solve a large problem, both companies we're excited to hear about our progress! We have working components of our MVP, including a robust user interface that works offline on desktop in addition to the browser, models to parse Zoom video and audio data into prompts for reasoning, and finally outputs to the user displaying conversation, relevant images, and more! We took a simple idea at first and scaled it to lots of innovative tracks with a global problem and pulled off parts of a solution in a short time! We enjoyed collaborating with each other, having all been from different places and meeting for this first time while at TreeHacks 📚 ### What we learned What kinds of needs or pain points exist in the education world and how technology can solve them What kinds of applications Edge AI can be useful in to solve problems How to integrate lots of technology with each other effectively to create a large yet useful and scalable app How to collaborate on a team with diverse skill sets, backgrounds, preferences, and perspectives How to manage time effectively and prioritize tasks and features given the complexity of each piece of our project (sacrifices had to be made at each step along the way). How to focus on the USER'S need before our own desires of what features we want to build (it's a tricky lesson to learn) ⏳ ### What's next for LinguaVista? We want to grow our user base and eventually support the ability for real people to chat with other real people! Another effective way to learn is by experimenting, such as practicing a language with a fluent speaker, asking experts questions, and having real conversations with great people. And what more effective way to learn than from someone in the place in question or with knowledge on the subject. Our mission for LinguaVista is to connect the world in ways not initially thought could be possible by the tech world, and with the opportunities in communication technology and Edge AI, we can leverage these incredible computing feats to educate our world for decades to come. Sources Kohnke, Lucas. "Edmodo: A Tool for Language Teaching and Learning." TESL-EJ, vol. 20, no. 3, Nov. 2016, pp. 1-8. ERIC, https://files.eric.ed.gov/fulltext/EJ1112894.pdf. ## README (from the GitHub repository) # LinguaVista - Language & Cultural Learning Platform See, speak, and connect with others around the world through AI-powered, immersive visual learning. ## Authors - Henry Shires - Brayton Lordianto - Raina Song - Abhushan Pradhan ## Made at TreeHacks 2025 Devpost: https://devpost.com/software/linguavista-language-cultural-learning-platform ## Detected evidence (automated analysis) Indexed codebase: 29 recognized source files, 27 KB. - CSS (language) — detected in the code - Express (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - React (technology) — detected in the code - TypeScript (language) — detected in the code - Ollama (technology) — claimed on Devpost, not found in the code - Python (language) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (38 of 38) ``` .gitignore .prettierrc frontend/.gitignore frontend/forge.config.js frontend/package.json frontend/public/index.html frontend/public/manifest.json frontend/public/robots.txt frontend/README.md frontend/src/App.tsx frontend/src/components/Navbar.tsx frontend/src/components/SearchBar.tsx frontend/src/context/SearchContext.tsx frontend/src/helpers/apiCalls.ts frontend/src/helpers/Lang.ts frontend/src/index.css frontend/src/index.html frontend/src/index.tsx frontend/src/main.js frontend/src/pages/Chat.tsx frontend/src/pages/CreateProfile.tsx frontend/src/pages/Home.tsx frontend/src/pages/Notes.tsx frontend/src/preload.js frontend/src/react-app-env.d.ts frontend/src/renderer.js frontend/tsconfig.json frontend/webpack.main.config.js frontend/webpack.renderer.config.js frontend/webpack.rules.js lingua-vista-express-api/.gitignore lingua-vista-express-api/app.js lingua-vista-express-api/controllers/user.js lingua-vista-express-api/models/userRooms.js lingua-vista-express-api/package.json lingua-vista-express-api/routes/image.js lingua-vista-express-api/routes/user.js README.md ``` ### Dependencies - frontend/package.json: @ant-design/icons@^5.6.1, @babel/core@^7.26.9, @babel/preset-react@^7.26.3, @babel/preset-typescript@^7.26.0, @electron-forge/cli@^7.6.1, @electron-forge/maker-deb@^7.6.1, @electron-forge/maker-rpm@^7.6.1, @electron-forge/maker-squirrel@^7.6.1, @electron-forge/maker-zip@^7.6.1, @electron-forge/plugin-auto-unpack-natives@^7.6.1, @electron-forge/plugin-fuses@^7.6.1, @electron-forge/plugin-webpack@^7.6.1, @electron/fuses@^1.8.0, @types/node@^16.18.126, @types/react@^19.0.8, @types/react-dom@^19.0.3, @vercel/webpack-asset-relocator-loader@1.7.3, antd@^5.24.0, babel-loader@^9.2.1, css-loader@^6.0.0, electron@34.2.0, electron-squirrel-startup@^1.0.1, node-loader@^2.0.0, react@^19.0.0, react-dom@^19.0.0, react-router-dom@^7.1.5, react-scripts@5.0.1, style-loader@^3.0.0, typescript@^4.9.5 - lingua-vista-express-api/package.json: axios@^1.7.9, body-parser@^1.20.3, cors@^2.8.5, express@^4.21.2 ### Recent commits (newest first) - Update README.md - Update README.md - Update homepage navigation - Remove old route - Merge pull request #4 from hcshires/uiux - Merge electron with new ui - Merge branch 'main' of https://github.com/hcshires/LinguaVista into uiux - Actually integrate electron this time - Notes and Search pages - Merge pull request #3 from hcshires/electron - added electron packaging - Fix warning - Barebones UI - Add skeleton frontend pages - New CRA - Merge pull request #2 from hcshires/chakra - UI start - Test out local llama and stable diffusion - Electron install - Create user form, express server, and room matching ## Key source files (fetched from GitHub, selected and truncated for size) ### lingua-vista-express-api/package.json ``` { "scripts": { "start": "node app.js" }, "dependencies": { "axios": "^1.7.9", "body-parser": "^1.20.3", "cors": "^2.8.5", "express": "^4.21.2" } } ``` ### frontend/package.json ``` { "name": "frontend", "productName": "frontend", "version": "1.0.0", "description": "My Electron application description", "main": ".webpack/main", "scripts": { "start": "npm i && react-scripts start", "electron": "npm i && electron-forge start", "build": "npm i && react-scripts build", "package": "npm i && electron-forge package", "make": "npm i && electron-forge make", "publish": "npm i && electron-forge publish", "lint": "echo \"No linting configured\"" }, "devDependencies": { "@babel/core": "^7.26.9", "@babel/preset-react": "^7.26.3", "@babel/preset-typescript": "^7.26.0", "@electron-forge/cli": "^7.6.1", "@electron-forge/maker-deb": "^7.6.1", "@electron-forge/maker-rpm": "^7.6.1", "@electron-forge/maker-squirrel": "^7.6.1", "@electron-forge/maker-zip": "^7.6.1", "@electron-forge/plugin-auto-unpack-natives": "^7.6.1", "@electron-forge/plugin-fuses": "^7.6.1", "@electron-forge/plugin-webpack": "^7.6.1", "@electron/fuses": "^1.8.0", "@types/node": "^16.18.126", "@types/react": "^19.0.8", "@types/react-dom": "^19.0.3", "@vercel/webpack-asset-relocator-loader": "1.7.3", "babel-loader": "^9.2.1", "css-loader": "^6.0.0", "electron": "34.2.0", "node-loader": "^2.0.0", "style-loader": "^3.0.0" }, "keywords": [], "author": { "name": "Henry Shires", "email": "hcsgeek@gmail.com" }, "license": "MIT", "dependencies": { "@ant-design/icons": "^5.6.1", "antd": "^5.24.0", "electron-squirrel-startup": "^1.0.1", "react": "^19.0.0", "react-dom": "^19.0.0", "react-router-dom": "^7.1.5", "react-scripts": "5.0.1", "typescript": "^4.9.5" }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] }, "build": { "extraResources": [ { "from": "./public/images", "to": "images" } ] } } ``` ### lingua-vista-express-api/app.js ```javascript const express = require("express"); const cors = require("cors"); const userRooms = require("./models/userRooms.js"); const user = require("./routes/user.js"); const image = require("./routes/image.js"); const app = express(); const port = 3010; app.listen(port, () => { console.log(`Server is running on port ${port}`); }); app.use(express.json()); app.use(cors()); app.use("/api/user", user); app.use("/api/image", image); ``` ### frontend/src/index.tsx ```typescript import React from "react"; import ReactDOM from "react-dom/client"; import "./index.css"; import App from "./App.tsx"; const root = ReactDOM.createRoot(document.body); root.render( ); ``` ### frontend/src/App.tsx ```typescript import React from "react"; import { ConfigProvider } from "antd"; import { Routes, Route, HashRouter } from "react-router-dom"; import { SearchContextProvider } from "./context/SearchContext.tsx"; import { ROUTES } from "./helpers/Lang.ts"; import Home from "./pages/Home.tsx"; import Chat from "./pages/Chat.tsx"; import Notes from "./pages/Notes.tsx"; const App: React.FC = () => ( } /> } /> } /> {/* Add other routes */} ); export default App; ``` ### frontend/src/main.js ```javascript import { app, BrowserWindow } from "electron"; import path from "node:path"; import started from "electron-squirrel-startup"; // Handle creating/removing shortcuts on Windows when installing/uninstalling. if (started) { app.quit(); } const createWindow = () => { // Create the browser window. const mainWindow = new BrowserWindow({ width: 800, height: 600, webPreferences: { preload: MAIN_WINDOW_PRELOAD_WEBPACK_ENTRY, }, autoHideMenuBar: true, }); // and load the index.html of the app. mainWindow.loadURL(MAIN_WINDOW_WEBPACK_ENTRY); // Open the DevTools. mainWindow.webContents.openDevTools(); }; // This method will be called when Electron has finished // initialization and is ready to create browser windows. // Some APIs can only be used after this event occurs. app.whenReady().then(() => { createWindow(); // On OS X it's common to re-create a window in the app when the // dock icon is clicked and there are no other windows open. app.on("activate", () => { if (BrowserWindow.getAllWindows().length === 0) { createWindow(); } }); }); // Quit when all windows are closed, except on macOS. There, it's common // for applications and their menu bar to stay active until the user quits // explicitly with Cmd + Q. app.on("window-all-closed", () => { if (process.platform !== "darwin") { app.quit(); } }); // In this file you can include the rest of your app's specific main process // code. You can also put them in separate files and import them here. ``` ### frontend/webpack.renderer.config.js ```javascript const rules = require('./webpack.rules'); rules.push({ test: /\.css$/, use: [{ loader: 'style-loader' }, { loader: 'css-loader' }], }); module.exports = { // Put your normal webpack config below here module: { rules, }, }; ``` ### frontend/webpack.main.config.js ```javascript module.exports = { /** * This is the main entry point for your application, it's the first file * that runs in the main process. */ entry: './src/main.js', // Put your normal webpack config below here module: { rules: require('./webpack.rules'), }, }; ``` ### frontend/webpack.rules.js ```javascript module.exports = [ // Add support for native node modules { // We're specifying native_modules in the test because the asset relocator loader generates a // "fake" .node file which is really a cjs file. test: /native_modules[/\\].+\.node$/, use: "node-loader", }, { test: /[/\\]node_modules[/\\].+\.(m?js|node)$/, parser: { amd: false }, use: { loader: "@vercel/webpack-asset-relocator-loader", options: { outputAssetBase: "native_modules", }, }, }, { test: /\.tsx?$/, use: { loader: "babel-loader", options: { exclude: /node_modules/, presets: ["@babel/preset-react", "@babel/preset-typescript"], }, }, }, ]; ``` ### frontend/forge.config.js ```javascript const { FusesPlugin } = require('@electron-forge/plugin-fuses'); const { FuseV1Options, FuseVersion } = require('@electron/fuses'); module.exports = { packagerConfig: { asar: true, }, rebuildConfig: {}, makers: [ { name: '@electron-forge/maker-squirrel', config: {}, }, { name: '@electron-forge/maker-zip', platforms: ['darwin'], }, { name: '@electron-forge/maker-deb', config: {}, }, { name: '@electron-forge/maker-rpm', config: {}, }, ], plugins: [ { name: '@electron-forge/plugin-auto-unpack-natives', config: {}, }, { name: '@electron-forge/plugin-webpack', config: { mainConfig: './webpack.main.config.js', renderer: { config: './webpack.renderer.config.js', entryPoints: [ { html: './src/index.html', js: './src/renderer.js', name: 'main_window', preload: { js: './src/preload.js', }, }, ], }, }, }, // Fuses are used to enable/disable various Electron functionality // at package time, before code signing the application new FusesPlugin({ version: FuseVersion.V1, [FuseV1Options.RunAsNode]: false, [FuseV1Options.EnableCookieEncryption]: true, [FuseV1Options.EnableNodeOptionsEnvironmentVariable]: false, [FuseV1Options.EnableNodeCliInspectArguments]: false, [FuseV1Options.EnableEmbeddedAsarIntegrityValidation]: true, [FuseV1Options.OnlyLoadAppFromAsar]: true, }), ], }; ``` [19 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]