HackStack

Project Info

GERAS: Guided Expedited Real-time Audio Screening

Devpost

This project did not submit a demo video on Devpost.

Inspiration

Phone scammers frequently target older adults by conducting financial, healthcare, and other forms of fraud over phone calls. Older individuals frequently prefer phone calls over digital customer service channels, and are especially vulnerable to this type of attack. Healthcare fraud is also especially prevalent as individuals over 65 automatically qualify for Medicare. To address this specific issue, our group developed GERAS, an AI-powered real-time scam detection tool.

What it does

GERAS uses a machine learning model to determine whether a phone call is malicious. For our demo, we compared S4 (Structured State Space for Sequence Modeling) with a fine-tuned BERT model. The S4 model generates an approximate reconstruction of the conversation using a very specialized latent space representation that gets updated in linear time. One key feature is that the sequence is never stored explicitly, which circumvents the limitations of a context window and of storage. A second key feature is that GERAS is capable of running inference online using far fewer parameters than a LLM.

How we built it

We used an A100 GPU to train and fine-tune our models, then we used Streamlit to build the frontend.

Challenges we ran into

One of the primary challenges was the limited availability of data for scam calls, often containing private or sensitive information shared by victims with scammers. To overcome this, we utilized recordings from organizations such as the FCC and other government agencies focused on consumer protection, supplemented by the generation of synthetic data.

Accomplishments we're proud of

Our structured state space approach does not require learning English or understanding the semantic meaning of a conversation. Yet, it can be employed for conversation classification, thereby reducing reliance on high-quality audio for accurate speech-to-text conversion.

What we learned

We gained insights into the complexities of addressing scam calls and the importance of innovative solutions that prioritize privacy and efficiency.

What's next

Our next steps include integrating GERAS with cell phones or applications with phone call functionality, such as WhatsApp, to extend its reach and effectiveness. These apps are our strategic next steps as many older adults use them frequently to contact relatives and friends.

Analysis

Compare with all teams

View

Metric

Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.

Technology

Found in codeClaimed only
  • CSSIn code
  • HTMLIn code
  • Hugging FaceIn code
  • JavaScriptIn code
  • PythonIn code
  • PyTorchIn code

6 of 6 appear in the indexed code.

AI coding agents

No AI coding agent signals were found in this repository.

Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.

Codebase size

Source size

56 KB

Source files

12

Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.

0 stars