# Project export: SpyLingo

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2024
- Tagline: Learn a foreign language by extracting information from targets
- Devpost: https://devpost.com/software/spylingo
- GitHub: https://github.com/riverliway/spylingo
- Video: https://www.youtube.com/embed/kwckRaQnAC4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 1 GitHub contributor(s) — River (17 commits)

## Devpost submission (written by the team)

### Overview

About Learning a foreign language can pose challenges, particularly without opportunities for conversational practice. Enter SpyLingo! Enhance your language proficiency by engaging in missions designed to extract specific information from targets. You select a conversation topic, and the spy agency devises a set of objectives for you to query the target about, thereby completing the mission. Users can choose their native language and the language they aim to learn. The website and all interfaces seamlessly translate into their native tongue, while missions are presented in the foreign language. Features Choose a conversation topic provided by the spy agency and it will generate a designated target and a set of objectives to discuss. Engage the target in dialogue in the foreign language on any subject! As you achieve objectives, they'll be automatically marked off your mission list. Witness dynamically generated images of the target, reflecting the topics they discuss, after each response. Enhance listening skills with automatically generated audio of the target's response. Translate the entire message into your native language for comprehension checks. Instantly translate any selected word within the conversation context, providing additional examples of its usage in the foreign language, which can be bookmarked for future review. Access hints for formulating questions about the objectives list to guide interactions with the target. Your messages are automatically checked for grammar and spelling, with explanations in your native language for correcting foreign language errors.

### How we built it

With the time constraint of the hackathon, this project was built entirely on the frontend of a web application. The TogetherAI API was used for all text and image generation and the ElevenLabs API was used for audio generation. The OpenAI API was used for detecting spelling and grammar mistakes.

### Challenges we ran into

The largest challenge of this project was building something that can work seamlessly in 812 different native-foreign language combinations. There was a lot of time spent on polishing the user experience to work with different sized text, word parsing, different punctuation characters, etc. Even more challenging was the prompt engineering required to ensure the AI would speak in the language it is supposed to. The chat models frequently would revert to English if the prompt was in English, even if the prompt specified the model should respond in a different language. As a result, there are over 800 prompts used, as each one has to be translated into every language supported during build time. There was also a lot of challenges in reducing the latency of the API responses to make for a pleasant user experience. After many rounds of performance optimizations, the app now effectively generates the text, audio, and images in perceived real time. Accomplishments that I'm proud of The biggest challenges also yielded the biggest accomplishments in my eyes. Building a chatbot that can be interacted with in any language and operates in real time by myself in the time limit was certainly no small task. I'm also exceptionally proud of the fact that I honestly think it's fun to play. I've had many projects that get dumped on a dusty shelf once completed, but the fact that I fully intend to keep using this after the hackathon to improve my language skills makes me very happy.

### What we learned

I had never used these APIs before beginning this hackathon, so there was quite a bit of documentation that I had to read to understand for how to correctly stream the text & audio generation.

### What's next

There are still more features that I'd like to add, like different types of missions for the user. I also think the image prompting can use some more work since I'm not as familiar with image generation. I would like to productionize this project and setup a proper backend & database for it. Maybe I'll set up a stripe integration and make it available for the public too!

## README (from the GitHub repository)

# spylingo
Learn foreign languages through spying on your targets and extracting information from them


## Detected evidence (automated analysis)

Indexed codebase: 30 recognized source files, 436 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- OpenAI (technology) — detected in the code
- React (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (34 of 34)

```
.gitignore
index.html
package.json
README.md
src/App.tsx
src/components/Chat/ChatBubble.css
src/components/Chat/ChatBubble.tsx
src/components/Chat/ChatPage.css
src/components/Chat/ChatPage.tsx
src/components/Chat/IntroAgent.css
src/components/Chat/IntroAgent.tsx
src/components/Chat/MissionAgent.tsx
src/components/ChooseLanguage/ChooseLanguage.css
src/components/ChooseLanguage/ChooseLanguage.tsx
src/components/CorrectionModal/CorrectionModal.tsx
src/components/HintModal/HintModal.tsx
src/components/Router/Router.tsx
src/components/SettingsModal/SettingsModal.css
src/components/SettingsModal/SettingsModal.tsx
src/context/apiContext.tsx
src/context/ChatContext.tsx
src/context/SettingsContext.tsx
src/index.css
src/main.tsx
src/utils/asyncTimeout.ts
src/utils/constants.ts
src/utils/languages.ts
src/utils/paginateArray.ts
src/utils/prompts.ts
src/utils/useAsyncEffect.ts
src/vite-env.d.ts
tsconfig.json
tsconfig.node.json
vite.config.ts
```

### Dependencies

- package.json: @ant-design/icons@^5.2.6, @types/react@^18.2.48, @types/react-dom@^18.2.18, @types/react-router-dom@^5.3.3, @vitejs/plugin-react@^4.2.1, antd@^5.13.2, dotenv@^16.3.1, openai@^4.28.0, react@^18.2.0, react-dom@^18.2.0, react-responsive@^9.0.2, react-router-dom@^6.21.3, together-ai-sdk@^0.0.7, ts-standard@^12.0.2, typescript@^5.2.2, vite@^5.0.11

### Recent commits (newest first)

- feat: improve streaming of audio
- feat: build grammar checker
- feat: add hint button
- feat: complete the quest system
- feat: show language names in native language
- feat: create better real time audio streams
- feat: add action items for each chat bubble
- feat: translate prompts into user's language
- feat: build UI for selecting missions
- feat: hook up elevenlabs api
- feat: dynamically generate response images
- refact: get rid of thinking image
- feat: set up introduction prompt for handler
- feat: hooked up chat to API
- feat: create contexts for holding information
- feat: build language selectors
- feat: init project
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
  "name": "spylingo",
  "version": "0.0.0",
  "description": "Learn foreign languages through spying on your targets and extracting information from them",
  "main": "index.js",
  "scripts": {
    "dev": "vite",
    "build": "tsc && vite build",
    "preview": "vite preview"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com:riverliway/spylingo.git"
  },
  "author": "River",
  "license": "ISC",
  "devDependencies": {
    "@types/react": "^18.2.48",
    "@types/react-dom": "^18.2.18",
    "@types/react-router-dom": "^5.3.3",
    "dotenv": "^16.3.1",
    "ts-standard": "^12.0.2",
    "typescript": "^5.2.2",
    "vite": "^5.0.11"
  },
  "dependencies": {
    "@ant-design/icons": "^5.2.6",
    "@vitejs/plugin-react": "^4.2.1",
    "antd": "^5.13.2",
    "openai": "^4.28.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-responsive": "^9.0.2",
    "react-router-dom": "^6.21.3",
    "together-ai-sdk": "^0.0.7"
  }
}

```

### src/main.tsx

```typescript
import ReactDOM from 'react-dom/client'
import App from './App'
import './index.css'

ReactDOM.createRoot(document.getElementById('root')!).render(
  <App />,
)

```

### src/App.tsx

```typescript
import React from 'react'
import { ConfigProvider, theme } from 'antd'
import { Router } from './components/Router/Router'
import { ApiProvider } from './context/apiContext'
import { SettingsProvider } from './context/SettingsContext'
import { ChatInfoProvider } from './context/ChatContext'

/**
 * The main component that wraps the entire application
 */
const App: React.FC = () => {
  return (
    <ConfigProvider theme={{ algorithm: theme.darkAlgorithm }}>
      <ApiProvider>
        <SettingsProvider>
          <ChatInfoProvider>
            <Router />
          </ChatInfoProvider>
        </SettingsProvider>
      </ApiProvider>
    </ConfigProvider>
  )
}

export default App

```

### index.html

```html
<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/vite.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>SpyLingo</title>
  </head>
  <body>
    <div id="root"></div>
    <script type="module" src="/src/main.tsx"></script>
  </body>
</html>

```

### vite.config.ts

```typescript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import 'dotenv/config'

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    react()
  ],
  define: {
    TOGETHER_API_KEY: JSON.stringify(process.env.TOGETHER_API_KEY),
    ELEVEN_LABS_KEY: JSON.stringify(process.env.ELEVEN_LABS_KEY),
    OPENAI_API_KEY: JSON.stringify(process.env.OPENAI_API_KEY)
  }
})

```

### src/vite-env.d.ts

```typescript
/// <reference types="vite/client" />

declare const TOGETHER_API_KEY: string
declare const ELEVEN_LABS_KEY: string
declare const OPENAI_API_KEY: string

```

### src/index.css

```css
:root {
  font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif;
  line-height: 1.5;
  font-weight: 400;

  color-scheme: light dark;
  color: rgba(255, 255, 255, 0.87);
  background-color: #242424;

  font-synthesis: none;
  text-rendering: optimizeLegibility;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  width: 100%;
}

a {
  font-weight: 500;
  color: #646cff;
  text-decoration: inherit;
}
a:hover {
  color: #535bf2;
}

body {
  width: 100%;
  margin: 0;
}

body > div {
  width: 100%;
}

h1 {
  font-size: 3.2em;
  line-height: 1.1;
}

button {
  border-radius: 8px;
  border: 1px solid transparent;
  padding: 0.6em 1.2em;
  font-size: 1em;
  font-weight: 500;
  font-family: inherit;
  background-color: #1a1a1a;
  cursor: pointer;
  transition: border-color 0.25s;
}
button:hover {
  border-color: #646cff;
}
button:focus,
button:focus-visible {
  outline: 4px auto -webkit-focus-ring-color;
}

@media (prefers-color-scheme: light) {
  :root {
    color: #213547;
    background-color: #ffffff;
  }
  a:hover {
    color: #747bff;
  }
  button {
    background-color: #f9f9f9;
  }
}

```

### src/utils/useAsyncEffect.ts

```typescript
import { DependencyList, useEffect } from 'react'

/**
 * The useEffect hook, but also runs for async functions
 * @param effect - The effect to run
 * @param deps - The dependencies to run the effect on
 */
export const useAsyncEffect = (effect: () => Promise<any>, deps: DependencyList): void => {
  useEffect(() => {
    void effect()
  }, deps)
}

```

### src/utils/asyncTimeout.ts

```typescript
/**
 * Waits for the timeout to expire, then runs the callback
 * Use this when needing to await the timeout:
 * `await asyncTimeout(func, 1000)`
 * @param timeout - number of milliseconds to wait
 * @param callback - the function to execute when the timeout is finished.
 * Acts as a blocking sleep function if no callback is provided
 */
export const asyncTimeout = async (timeout: number, callback?: () => void): Promise<void> => {
  return await new Promise(resolve => {
    setTimeout(() => {
      callback?.()
      resolve()
    }, timeout)
  })
}

```

### src/utils/paginateArray.ts

```typescript

/**
 * Paginates any array into subarrays. Each subarray will have length of
 * exactly `pageLength`, except the last subarray, which may have less.
 * @param array - the array to paginate
 * @param pageLength - the length of each subarray, must be a positive number
 * @returns a list of subarrays
 */
export const paginateArray = <T>(array: T[], pageLength: number): T[][] => {
  if (pageLength <= 0) return [array]
  if (array.length === 0) return [[]]
  const len = Math.floor(pageLength)

  const ret = []
  for (let i = 0; i < array.length / len; i++) {
    ret.push(array.slice(i * len, (i + 1) * len))
  }

  return ret
}

```

[18 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]