# Project export: Hello

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Video chat web app powered with real-time ASL transcription.
- Devpost: https://devpost.com/software/hello-gfsy3l
- GitHub: not linked
- Result: winner (Deepgram: Deepgram Voice Agent Quest)
- Team: contributor stats unavailable

## Devpost submission (written by the team)

### Inspiration

Only a small percentage of Americans use ASL as their main form of daily communication. Hence, no one notices when ASL-first speakers are left out of using FaceTime, Zoom, or even iMessage voice memos. This is a terrible inconvenience for ASL-first speakers attempting to communicate with their loved ones, colleagues, and friends. There is a clear barrier to communication between those who are deaf or hard of hearing and those who are fully-abled. We created Hello as a solution to this problem for those experiencing similar situations and to lay the ground work for future seamless communication. On a personal level, Brandon's grandma is hard of hearing, which makes it very difficult to communicate. In the future this tool may be their only chance at clear communication.

### What it does

Expectedly, there are two sides to the video call: a fully-abled person and a deaf or hard of hearing person. For the fully-abled person: Their speech gets automatically transcribed in real-time and displayed to the end user Their facial expressions and speech get analyzed for sentiment detection For the deaf/hard of hearing person: Their hand signs are detected and translated into English in real-time The translations are then cleaned up by an LLM and displayed to the end user in text and audio Their facial expressions are analyzed for emotion detection

### How we built it

Our frontend is a simple React and Vite project. On the backend, websockets are used for real-time inferencing. For the fully-abled person, their speech is first transcribed via Deepgram, then their emotion is detected using HumeAI. For the deaf/hard of hearing person, their hand signs are first translated using a custom ML model powered via Hyperbolic, then these translations are cleaned using both Google Gemini and Hyperbolic. Hume AI is used similarly on this end as well. Additionally, the translations are communicated back via text-to-speech using Cartesia/Deepgram.

### Challenges we ran into

Custom ML models are very hard to deploy (Credits to https://github.com/hoyso48/Google---American-Sign-Language-Fingerspelling-Recognition-2nd-place-solution) Websockets are easier said than done Spotty wifi

### Accomplishments we're proud of

Learned websockets from scratch Implemented custom ML model inferencing and workflows More experience in systems design

### What's next

Faster, more accurate ASL model. More scalability and maintainability for the codebase.

## README (from the GitHub repository)

No README available.

## Detected evidence (automated analysis)

No repository was indexed for this project. Claimed technologies below could not be checked against code.
- Google Gemini (technology) — claimed on Devpost, not found in the code
- React (technology) — claimed on Devpost, not found in the code

## Codebase structure

No repository index available.

## Key source files

No repository index available; no source files included.