# Project export: LanguaLine

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Are you anxious about speaking a foreign language or don't know where to start? Practice your conversational language skills and get real-time feedback with LanguaLine!
- Devpost: https://devpost.com/software/langualine
- GitHub: https://github.com/SrimanCode/CalHacks-11.0
- Video: https://www.youtube.com/embed/sJRIEn7pzM4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Result: winner (Vapi: Show us your Voice AI)
- Team: 4 GitHub contributor(s) — aishse (48 commits), Sriman Ujhanthachhen (26 commits), sharma-pari (14 commits), JDLiang100 (4 commits)

## Devpost submission (written by the team)

### Inspiration

With a variety of language learning resources out there, we set out to create a tool that can help us practice a part of language that keeps things flowing -- conversation! LanguaLine aims to empower users to speak their foreign language by helping them develop their conversational skills.

### What it does

We wanted to create an interface that can help users practice speaking a foreign language. Through LanguaLine, users can: Select a language they wish to practice speaking in Select how "motivating" they want their Mentor to be (Basic is normal, Extra Motivation is a tough love approach). Enter their phone number and receive a phone call from our AI Mentor Answer questions posed by the Mentor Receive real-time feedback about their performance View a transcript and summary of the call after the conversation is completed View a generalized report on user's language strengths and weaknesses across all conversations

### How we built it

We used React.js for our frontend, and Firebase for our database. To style our components, we utilized TailwindCSS and React MaterialUI. Our backend system is comprised of Node.js and Express.js, which we use to make calls to Google's Gemini model. To create, tune, and prompt engineer our AI Mentor, we used the VAPI.ai API. Our transcriber model is Deepgram's nova-2 multi and our model is gpt-4o-mini provided by Open.AI. We are also using Gemini to implement Retrieval-Augmented Generation (RAG). We use past call transcripts and summaries to train and optimize our model. This training data is maintained in our Firebase database. In addition, this feature analyzes users' call transcripts and generates a report identifying strengths and weaknesses in their speaking skills.

### Challenges we ran into

Our biggest challenge was understanding the VAPI documentation, as it was our first time working with a voice AI API. We had to make a few changes to our project stack to accommodate for VAPI, as we could only make client-side API calls. Since the majority of our team has limited experience working with LLMs and voice AI Agents, we faced some difficulties prompt engineering our Mentor, requiring us to tweak various model parameters and experiment through VAPI's dashboard.

### Accomplishments we're proud of

The turning point in our development process was when we were able to start conversing with our Mentor. After this was solidified, our project trajectory only went upwards. We're proud of the fact we were able to turn this idea into an operational and functional application.

### What we learned

The team behind LanguaLine had a variety of skill levels; for some, this was their first project using this tech stack, while for others, this was familiar. Some of us mastered the ability to send API calls and parse JSON data. Some of us also learned how to prompt engineer for a particular language choice. There were lessons being learned all throughout the 36 hours of development, which helped us feel connected to the project and motivated to keep creating.

### What's next

Our biggest goal is to deploy and market this project. Being language learners ourselves, having a service like LanguaLine is invaluable to making progress toward achieving fluency. In addition, this increases accessibility for language learners by encompassing a wide range of supported languages and providing customizable support. Our project aims to support all languages. Due to our lack of control regarding the accuracy across various languages within LLMs, this feature needs more testing and tuning to be perfected. We plan to offer more variation in the Mentors we offer. Right now, we only offer Mentors based off of a language choice and motivation level. In the future, we plan to include language difficulties, personalities, a wider variety of supported languages, custom prompts, and scheduled calling. We also plan to offer improvement plans for grammar, pronunciation, and vocabulary, as well as a scoring system for users' performances during Mentor sessions.

## README (from the GitHub repository)

# Callhacks 11.0 Project: LanguaLine
Team: Pari, David, Sriman, Anishka

**Table number: 30**

## Inspiration
With a variety of language learning resources out there, we set out to create a tool that can help us practice a part of language that keeps things flowing -- conversation! LanguaLine aims to empower users to speak their foreign language by helping them develop their conversational skills. 

## What it does
We wanted to create an interface that can help users practice speaking a foreign language. Through LanguaLine, users can: 
- Select a language they wish to practice speaking in
- Select how "motivating" they want their Mentor to be (Basic is normal, Extra Motivation is a tough love approach). 
- Enter their phone number and receive a call from our AI Mentor
- Answer questions posed by the Mentor 
- Receive real-time feedback about their performance and a generalized report on strengths and weakensses
- View a transcript and summary of the call after the conversation is completed

## How we built it
We used ```React.js``` for our frontend, and ```Firebase``` for our database. To style our components, we utilized ```TailwindCSS``` and ```React MaterialUI```. 

To create, tune, and prompt engineer our AI Mentor, we used the **VAPI.ai** API. Our **transcriber model** is Deepgram's **nova-2 multi** and our **model** is **gpt-4o-mini** provided by **Open.AI**.  Our Mentor's voice is **Alloy**, also provided by **Open.AI**. 

We are also using **Gemini** to analyze users' call transcripts and generate a report identifying strengths and weaknesses in their speaking skills. 

## Challenges we ran into
Our biggest challenge was understanding the VAPI documentation, as it was our first time working with a voice AI API. We had to make a few changes to our project stack to accommodate for VAPI, as we could only make client-side API calls. unlike our original plan of using ```Express.js```.  

Since the majority of our team has limited experience working with LLMs and voice AI Agents, we faced some difficulties prompt engineering our Mentor, requiring us to tweak various model parameters and experiment through VAPI's dashboard.

## Accomplishments that we're proud of
The turning point in our development process was when we were able to start conversing with our Mentor. After this was solidified, our project trajectory only went upwards. We're proud of the fact we were able to turn this idea into an operational and functional application. 

## What we learned
The team behind LanguaLine had a variety of skill levels; for some, this was their first project using this tech stack, while for others, this was familiar. Some of us mastered the ability to send API calls and parse JSON data. Some of us also learned how to prompt engineer for a particular language choice. There were lessons being learned all throughout the 36 hours of development, which helped us feel connected to the project and motivated to keep creating. 

## What's next for LanguaLine
We plan to offer **more variation** in the Mentors we offer. Right now, we only offer Mentors based off of a language choice and motivation level. In the future, we plan to include language difficulties, personalities, greater language support, custom prompts, and scheduled calling. 

We also plan to offer improvement plans for grammar, pronunciation, and vocabulary, as well as a scoring system for users' performances during Mentor sessions. 


## Detected evidence (automated analysis)

Indexed codebase: 24 recognized source files, 43 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- Firebase (technology) — detected in the code
- Google Gemini (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Node.js (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (34 of 34)

```
.DS_Store
.gitignore
backend/.gitignore
backend/package.json
backend/server.js
calhacks11.0/.env.local
calhacks11.0/.gitignore
calhacks11.0/database.rules.json
calhacks11.0/package.json
calhacks11.0/public/index.html
calhacks11.0/public/manifest.json
calhacks11.0/public/robots.txt
calhacks11.0/README.md
calhacks11.0/src/App.css
calhacks11.0/src/App.js
calhacks11.0/src/App.test.js
calhacks11.0/src/Components/callsummary.js
calhacks11.0/src/Components/fire.css
calhacks11.0/src/Components/FireComponent.js
calhacks11.0/src/Components/GetLLMData.js
calhacks11.0/src/Components/ModeToggle.js
calhacks11.0/src/Components/NavBar.js
calhacks11.0/src/Components/outboundcalls.js
calhacks11.0/src/Components/PhoneInput.js
calhacks11.0/src/Components/UserProgress.js
calhacks11.0/src/History.js
calhacks11.0/src/index.css
calhacks11.0/src/index.js
calhacks11.0/src/MainPage.js
calhacks11.0/src/progress.js
calhacks11.0/src/reportWebVitals.js
calhacks11.0/src/setupTests.js
calhacks11.0/tailwind.config.js
README.md
```

### Dependencies

- backend/package.json: @google/generative-ai@^0.21.0, @vapi-ai/web@^2.1.4, axios@^1.7.7, cors@^2.8.5, dotenv@^16.4.5, express@^4.21.1
- calhacks11.0/package.json: @clerk/clerk-react@^5.12.0, @emotion/react@^11.13.3, @emotion/styled@^11.13.0, @fontsource/roboto@^5.1.0, @mui/material@^6.1.4, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, axios@^1.7.7, dotenv@^16.4.5, firebase@^10.14.1, imask@^7.6.1, react@^18.3.1, react-dom@^18.3.1, react-imask@^7.6.1, react-router-dom@^6.27.0, react-script@^2.0.5, react-scripts@5.0.1, tailwindcss@^3.4.14, web-vitals@^2.1.4

### Recent commits (newest first)

- changes
- changes
- changes
- changes
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Create README.md
- frontend
- multilingual support
- Merge branch 'main' of https://github.com/SrimanCode/CalHacks-11.0
- frontend and backfeeding
- Merge branch 'main' of https://github.com/SrimanCode/CalHacks-11.0
- Merge branch 'main' of https://github.com/SrimanCode/CalHacks-11.0
- changes
- Update .gitignore
- Removing ignored filed
- removing ignored files

## Key source files (fetched from GitHub, selected and truncated for size)

### backend/package.json

```
{
  "name": "backend",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@google/generative-ai": "^0.21.0",
    "@vapi-ai/web": "^2.1.4",
    "axios": "^1.7.7",
    "cors": "^2.8.5",
    "dotenv": "^16.4.5",
    "express": "^4.21.1"
  }
}

```

### calhacks11.0/package.json

```
{
  "name": "calhacks11.0",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@clerk/clerk-react": "^5.12.0",
    "@emotion/react": "^11.13.3",
    "@emotion/styled": "^11.13.0",
    "@fontsource/roboto": "^5.1.0",
    "@mui/material": "^6.1.4",
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "axios": "^1.7.7",
    "dotenv": "^16.4.5",
    "firebase": "^10.14.1",
    "imask": "^7.6.1",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-imask": "^7.6.1",
    "react-router-dom": "^6.27.0",
    "react-script": "^2.0.5",
    "react-scripts": "5.0.1",
    "tailwindcss": "^3.4.14",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### backend/server.js

```javascript
const { GoogleGenerativeAI } = require("@google/generative-ai");

const express = require("express");
const axios = require("axios");
const cors = require("cors");
require("dotenv").config();

//middleware
const app = express();
app.use(cors());
app.use(express.json());
const port = 5000;
const genAI = new GoogleGenerativeAI(process.env.REACT_APP_GEMINI_API);
const model = genAI.getGenerativeModel({ model: "gemini-1.5-flash" });

app.post("/api/generate", async (req, res) => {
  const prompt = `
    You are an AI assistant helping a user learn a new language. Below is a transcript of a conversation between the user and the AI. Analyze the conversation and provide feedback on the user's progress in learning the language. Your feedback should include two sections: 
    1. "strongPoints" – highlight the areas where the user is performing well in their language learning.
    2. "progressPoints" – suggest specific areas where the user can improve or focus to make further progress in learning the language.

    Format your response in JSON, like this:
    {
      "response": {
        "strongPoints": [/* list of strong points */],
        "progressPoints": [/* list of areas for improvement */]
      }
    }
    `;

  try {
    const { transcript } = req.body;
    const result = await model.generateContent([prompt, transcript]);
    if (result && result.response && result.response.text) {
      res.send(result.response.text());
    } else {
      res.send("Result format doesn't match expected structure.");
    }
  } catch (error) {
    console.error("Error generating content:", error);
  }
});

app.listen(port, () => {
  console.log(`Server is running on port ${port}`);
});

```

### calhacks11.0/src/App.js

```javascript
import {
  SignedIn,
  SignedOut,
  SignIn,
  UserButton,
  useUser,
} from "@clerk/clerk-react";
import { useEffect } from "react";
import { useNavigate } from "react-router-dom";

export default function App() {
  const { isSignedIn } = useUser();
  const navigate = useNavigate();

  useEffect(() => {
    if (isSignedIn) {
      // Redirect to MainPage when user is signed in
      navigate("/main");
    }
  }, [isSignedIn, navigate]);

  return (
    <div className="flex justify-center items-center min-h-screen bg-gray-100">
      <header className="w-full max-w-md p-6 bg-white shadow-md rounded-lg">
        <SignedOut>
          <div className="flex justify-center">
            <SignIn />
          </div>
        </SignedOut>
        <SignedIn>
          <div className="flex justify-center">
            <UserButton />
          </div>
        </SignedIn>
      </header>
    </div>
  );
}

```

### calhacks11.0/src/index.js

```javascript
import React from "react";
import ReactDOM from "react-dom/client";
import "./index.css";
import App from "./App";
import reportWebVitals from "./reportWebVitals";
import "@fontsource/roboto/300.css";
import "@fontsource/roboto/400.css";
import "@fontsource/roboto/500.css";
import "@fontsource/roboto/700.css";
import { ClerkProvider } from "@clerk/clerk-react";
import { BrowserRouter as Router, Route, Routes } from "react-router-dom";
import MainPage from "./MainPage";
import History from "./History";
import Progress from "./progress";

const PUBLISHABLE_KEY =
  "pk_test_b2JsaWdpbmctcmVwdGlsZS0yMy5jbGVyay5hY2NvdW50cy5kZXYk";

if (!PUBLISHABLE_KEY) {
  throw new Error("Missing Publishable Key");
}

const root = ReactDOM.createRoot(document.getElementById("root"));
root.render(
  <React.StrictMode>
    <ClerkProvider publishableKey={PUBLISHABLE_KEY} afterSignOutUrl="/">
      <Router>
        <Routes>
          <Route path="/" element={<App />} />
          <Route path="/main" element={<MainPage />} />
          <Route path="/history" element={<History />} />
          <Route path="/progress" element={<Progress />} />
        </Routes>
      </Router>
    </ClerkProvider>
  </React.StrictMode>
);

reportWebVitals();

```

### calhacks11.0/tailwind.config.js

```javascript
/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    "./src/**/*.{js,jsx,ts,tsx}", // Adjust this to where your components are located
    "./public/index.html",
  ],
  theme: {
    extend: {
      animation: {
        shake: "shake 1s cubic-bezier(.36,.07,.19,.97) both",
        pulseone: "scale 0.5s 6s alternate",
        fall: "fall 2s forwards",
      },
      keyframes: {
        shake: {
          "10%, 90%": {
            transform: "translate3d(-1px, 0, 0)",
          },
          "20%, 80%": {
            transform: "translate3d(2px, 0, 0)",
          },
          "30%, 50%, 70%": {
            transform: "translate3d(-4px, 0, 0)",
          },
          "40%, 60%": {
            transform: "translate3d(4px, 0, 0)",
          },
        },
        scale: {
          "0%": { transform: "scale(1)" },
          "100%": { transform: "scale(1.02)" }, // Scale up slightly
        },
        fall: {
          "0%": { transform: "translateY(-100px)", opacity: "1" },
          "100%": { transform: "translateY(100vh)", opacity: "0" },
        },
      },
    },
  },
  plugins: [],
};

```

### calhacks11.0/src/App.css

```css
@tailwind base;
@tailwind components;
@tailwind utilities;



```

### calhacks11.0/src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### calhacks11.0/src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### calhacks11.0/src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

[14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]