# Project export: SerenityHelp
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: UC Berkeley AI Hackathon 2025
- Tagline: Fetch.ai sentiment agent processes live audio/text calls, detects crisis language and urgency, and integrates via Agentverse to prioritize and route high-need callers for faster support.
- Devpost: https://devpost.com/software/serenityhelp
- GitHub: https://github.com/kajenthavaraj/serenityhelp
- Demo: https://crisishotline.onrender.com/
- Team: 3 GitHub contributor(s) — kajenthavaraj (16 commits), Martin Ha (6 commits), Roshan Sanjeev (5 commits)
## Devpost submission (written by the team)
### Inspiration
Last year, a friend of mine was going through a mental health crisis. We called our school-sponsored helpline only to be left on hold for 20 minutes before we decided to hang up. Researching the issue, we found that helplines can take up to 15 minutes to connect calls during peak hours.
### What it does
You can call our helpline yourself at (765) 245-8252 We wanted to build a tool to allow helplines to let AI voice agent handle calls during peak traffic where human reps aren't available. Instead of being left on hold, callers can talk to a voice AI agent that can triage the call's urgency, allowing human agents to be more quickly connected to high-risk callers while having the AI voice agent give callers someone to talk to while they wait; potentially de-escalating a fatal crisis.
### How we built it
We used the Vapi framework to process calls and embed agentic tools to display on our frontend dashboard. -Vapi hosts 4 tools - Deepgram(which converts voice to text - Fetch.ai(our sentiment analysis agent that takes in both the audio recording and text transcript and creates a sentiment analysis in 4 categories to be used to update profiles on the dashboard -Groq is used for its fast speeds needed to process and respond to text input from the user to power our Live AI Phone call Voice Agent -LMNT takes the script Groq has written and converts from text to speech to communicate the message back the user
### Challenges we ran into
Needed to create a custom multimodal sentiment analysis model to triage our helpline's calls. We initially struggled with a complex backend setup and infrastructure issues while hosting our model. To streamline deployment, we turned to Fetch.ai—hosting the model there and integrating it into our app via a simple API call.
### Accomplishments we're proud of
Be able to detect sentiment over the audio call to be able to quantify self-harm, homicidal, psychosis, and distress ratings Live display of each call with transcript, distress ratings, recording, and user information
### What we learned
How to use Vapi to host voice AI calls Deploying a model for sentiment analysis Using Fetch.ai to host multimodal agents
### What's next
Create a login auth to allow call center agents to sign up automatically Vapi Vapi was the backbone of our voice AI infrastructure. It enabled us to easily manage real-time, bidirectional audio conversations between callers and our AI agent. Through Vapi, we connected four core tools into one cohesive pipeline: Deepgram for speech-to-text transcription of live calls Groq to rapidly process transcripts and generate empathetic, context-aware responses using LLMs LMNT to convert Groq's outputs into humanlike speech, enabling natural voice interactions Fetch.ai to run our custom multimodal sentiment analysis agent, which takes in both text and audio to assess crisis indicators in real time Vapi made it possible to stitch all of these components together seamlessly, handling call flow logic and latency requirements while giving us the flexibility to plug in specialized tools at each step. Without Vapi, building a fully functional AI voice agent in under 12 hours wouldn’t have been feasible. Fetch AI We used Fetch.ai to deploy and host our custom sentiment analysis agent, which plays a critical role in triaging mental health crisis calls. The agent ingests both the audio recording and text transcript of each call and returns a real-time analysis across four key emotional dimensions: self-harm, homicidal ideation, psychosis, and distress. Initially, we struggled with complex infrastructure when trying to host this model ourselves. Fetch.ai’s Agentverse platform made it easy to deploy the agent and expose it via a simple API call, dramatically simplifying our backend. Fetch.ai also allowed us to build a multimodal pipeline, enabling richer, context-aware insights by analyzing both voice tone and spoken content. These insights update our frontend dashboard live, helping human agents prioritize and respond faster to the most urgent calls. Groq Groq was essential to powering the real-time intelligence behind our AI voice agent. We used Groq to run our LLM-based reasoning system that generates empathetic, context-aware responses to the caller’s input during live phone conversations. What set Groq apart was its blazing-fast inference speed—crucial for maintaining a natural, uninterrupted dialogue between the caller and the AI. In a crisis setting, even a few seconds of delay can feel like forever. Groq’s ultra-low latency allowed us to process speech-to-text inputs and return meaningful responses in real time, without awkward pauses or delays. Thanks to Groq, we were able to maintain fluid, high-quality conversations that feel responsive and human-like—an essential part of building trust and de-escalating crises through voice AI.
## README (from the GitHub repository)
# SerenityHelp
## Inspiration
Last year, a friend of mine was going through a mental health crisis. We called our school-sponsored helpline only to be left on hold for 20 minutes before we decided to hang up.
Researching the issue, we found that helplines can take up to 15 minutes to connect calls during peak hours.
## What it does
You can call our helpline yourself at (765) 245-8252
We wanted to build a tool to allow helplines to let AI voice agent handle calls during peak traffic where human reps aren't available. Instead of being left on hold, callers can talk to a voice AI agent that can triage the call's urgency, allowing human agents to be more quickly connected to high-risk callers while having the AI voice agent give callers someone to talk to while they wait; potentially de-escalating a fatal crisis.
## How we built it
We used the Vapi framework to process calls and embed agentic tools to display on our frontend dashboard.
-Vapi hosts 4 tools
- Deepgram(which converts voice to text
- Fetch.ai(our sentiment analysis agent that takes in both the audio recording and text transcript and creates a sentiment analysis in 4 categories to be used to update profiles on the dashboard
-Groq is used for its fast speeds needed to process and respond to text input from the user to power our Live AI Phone call Voice Agent
-LMNT takes the script Groq has written and converts from text to speech to communicate the message back the user
## Challenges we ran into
- Needed to create a custom multimodal sentiment analysis model to triage our helpline's calls.
- We initially struggled with a complex backend setup and infrastructure issues while hosting our model. To streamline deployment, we turned to Fetch.ai—hosting the model there and integrating it into our app via a simple API call.
## Accomplishments that we're proud of
- Be able to detect sentiment over the audio call to be able to quantify self-harm, homicidal, psychosis, and distress ratings
- Live display of each call with transcript, distress ratings, recording, and user information
## What we learned
- How to use Vapi to host voice AI calls
- Deploying a model for sentiment analysis
- Using Fetch.ai to host multimodal agents
## What's next for SerenityHelp
- Create a login auth to allow call center agents to sign up automatically
## Vapi
Vapi was the backbone of our voice AI infrastructure. It enabled us to easily manage real-time, bidirectional audio conversations between callers and our AI agent. Through Vapi, we connected four core tools into one cohesive pipeline:
Deepgram for speech-to-text transcription of live calls
Groq to rapidly process transcripts and generate empathetic, context-aware responses using LLMs
LMNT to convert Groq's outputs into humanlike speech, enabling natural voice interactions
Fetch.ai to run our custom multimodal sentiment analysis agent, which takes in both text and audio to assess crisis indicators in real time
Vapi made it possible to stitch all of these components together seamlessly, handling call flow logic and latency requirements while giving us the flexibility to plug in specialized tools at each step. Without Vapi, building a fully functional AI voice agent in under 12 hours wouldn’t have been feasible.
## Fetch AI
We used Fetch.ai to deploy and host our custom sentiment analysis agent, which plays a critical role in triaging mental health crisis calls. The agent ingests both the audio recording and text transcript of each call and returns a real-time analysis across four key emotional dimensions: self-harm, homicidal ideation, psychosis, and distress.
Initially, we struggled with complex infrastructure when trying to host this model ourselves. Fetch.ai’s Agentverse platform made it easy to deploy the agent and expose it via a simple API call, dramatically simplifying our backend.
Fetch.ai also allowed us to build a multimodal pipeline, enabling richer, context-aware insights by analyzing both voice tone and spoken content. These insights update our frontend dashboard live, helping human agents prioritize and respond faster to the most urgent calls.
## Groq
Groq was essential to powering the real-time intelligence behind our AI voice agent. We used Groq to run our LLM-based reasoning system that generates empathetic, context-aware responses to the caller’s input during live phone conversations.
What set Groq apart was its blazing-fast inference speed—crucial for maintaining a natural, uninterrupted dialogue between the caller and the AI. In a crisis setting, even a few seconds of delay can feel like forever. Groq’s ultra-low latency allowed us to process speech-to-text inputs and return meaningful responses in real time, without awkward pauses or delays.
Thanks to Groq, we were able to maintain fluid, high-quality conversations that feel responsive and human-like—an essential part of building trust and de-escalating crises through voice AI.
## Detected evidence (automated analysis)
Indexed codebase: 25 recognized source files, 89 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Flask (technology) — claimed on Devpost, not found in the code
- Hugging Face (technology) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (35 of 35)
```
.gitignore
backend/.gitkeep
backend/app2.py
backend/SentimentAgent/analyze_emotions.py
backend/SentimentAgent/private_keys.json
backend/SentimentAgent/render.yaml
backend/SentimentAgent/requirements_server.txt
backend/SentimentAgent/requirements.txt
backend/SentimentAgent/send_request.py
backend/SentimentAgent/start.sh
backend/test_websocket.py
backend/test.py
frontend/.gitkeep
frontend/App.tsx
frontend/components/CallDetailsSidebar.tsx
frontend/components/CallLogTile.tsx
frontend/components/ConnectionStatus.tsx
frontend/components/Dashboard.tsx
frontend/components/pages/CallTransfer.tsx
frontend/contexts/CallContext.tsx
frontend/index.css
frontend/index.html
frontend/index.tsx
frontend/package.json
frontend/postcss.config.js
frontend/README.md
frontend/services/websocketService.ts
frontend/tailwind.config.js
frontend/tsconfig.json
frontend/tsconfig.node.json
frontend/types.d.ts
frontend/utils/mockData.ts
frontend/vite.config.ts
LICENSE
README.md
```
### Dependencies
- backend/SentimentAgent/requirements.txt: librosa@>=0.10.0, numpy@>=1.24.0, pydantic@>=2.0.0, pydub@>=0.25.0, python-speech-features@>=0.6.0, pyworld@>=0.3.0, scipy@>=1.10.0, soundfile@>=0.12.0, uagents@>=0.13.0, uagents-ai-engine@>=0.4.0, uvloop@>=0.17.0
- frontend/package.json: @types/react@^18.0.28, @types/react-dom@^18.0.11, @types/socket.io-client@^1.4.36, @vitejs/plugin-react@^3.1.0, autoprefixer@^10.4.14, lucide-react@^0.263.1, postcss@^8.4.21, react@^18.2.0, react-dom@^18.2.0, react-router-dom@^6.8.0, socket.io-client@^4.8.1, tailwindcss@^3.2.7, typescript@^4.9.5, vite@^4.1.0
### Recent commits (newest first)
- Update README.md
- Merge pull request #1 from kajenthavaraj/martin_dev
- good enough
- Merge branch 'main' of https://github.com/kajenthavaraj/serenityhelp
- Normalized scores
- rename
- Test file for vapi specifically
- Finished connecting backend to frontend
- New code for app2.py
- Added websocket file to frontend
- Created calldata class and function to make call to huggingface model
- Merge branch 'main' of https://github.com/kajenthavaraj/serenityhelp
- Created dummy websocket service to create new tiles
- Merge branch 'main' of https://github.com/kajenthavaraj/serenityhelp
- Initial Flask API setup; only live transcript so far
- Merge branch 'main' of https://github.com/kajenthavaraj/serenityhelp
- Completed resolve button
- deployed sentiment agent files for render
- deleted testfiles
- Merge branch 'main' of https://github.com/kajenthavaraj/serenityhelp
## Key source files (fetched from GitHub, selected and truncated for size)
### frontend/package.json
```
{
"name": "serenityhelp-frontend",
"version": "1.0.0",
"description": "AI-Powered Support Center Frontend",
"main": "index.tsx",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview",
"start": "vite"
},
"dependencies": {
"@types/socket.io-client": "^1.4.36",
"lucide-react": "^0.263.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-router-dom": "^6.8.0",
"socket.io-client": "^4.8.1"
},
"devDependencies": {
"@types/react": "^18.0.28",
"@types/react-dom": "^18.0.11",
"@vitejs/plugin-react": "^3.1.0",
"autoprefixer": "^10.4.14",
"postcss": "^8.4.21",
"tailwindcss": "^3.2.7",
"typescript": "^4.9.5",
"vite": "^4.1.0"
}
}
```
### backend/SentimentAgent/requirements.txt
```
# requirements_testing.txt - Additional packages for testing
uagents>=0.13.0
uagents-ai-engine>=0.4.0
pydantic>=2.0.0
uvloop>=0.17.0
# Audio processing for tonality analysis
librosa>=0.10.0
numpy>=1.24.0
scipy>=1.10.0
soundfile>=0.12.0
# For testing with different audio formats
pydub>=0.25.0
# Optional: For more advanced audio analysis
python-speech-features>=0.6.0
pyworld>=0.3.0
```
### frontend/index.tsx
```typescript
import './index.css'
import React from "react";
import { render } from "react-dom";
import { App } from "./App";
render(, document.getElementById("root"));
```
### frontend/App.tsx
```typescript
import React, { useEffect } from 'react'
import { BrowserRouter, Routes, Route } from 'react-router-dom'
import Dashboard from './components/Dashboard'
import CallTransfer from './components/pages/CallTransfer'
import { CallProvider } from './contexts/CallContext'
import { websocketService } from './services/websocketService'
export function App() {
useEffect(() => {
// Initialize WebSocket connection when app starts
console.log('🚀 Initializing WebSocket connection...')
websocketService.connect()
// Cleanup on unmount
return () => {
websocketService.disconnect()
}
}, [])
return (
} />
} />
)
}
```
### frontend/postcss.config.js
```javascript
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
```
### frontend/vite.config.ts
```typescript
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
export default defineConfig({
plugins: [react()],
server: {
port: 3000,
open: true
}
})
```
### frontend/tailwind.config.js
```javascript
/** @type {import('tailwindcss').Config} */
export default {
content: [
"./index.html",
"./**/*.{js,ts,jsx,tsx}",
"./components/**/*.{js,ts,jsx,tsx}",
"./utils/**/*.{js,ts,jsx,tsx}"
],
theme: {
extend: {},
},
plugins: [],
}
```
### frontend/index.html
```html
SerenityHelp - AI-Powered Support Center
```
### frontend/types.d.ts
```typescript
declare module '*.css' {
const content: { [className: string]: string };
export default content;
}
interface Call {
id: number;
callerName: string;
phoneNumber: string;
duration: string;
status: 'connected-to-agent' | 'connected-to-911' | 'in-progress' | 'completed';
topic: string;
summary: string;
priority: 'Emergency' | 'High Priority' | 'Normal' | 'Low Priority';
transcript: string;
riskAssessment: {
selfHarm: number;
distress: number;
homicidal: number;
psychosis: number;
};
date?: string;
time?: string;
}
```
### frontend/index.css
```css
/* PLEASE NOTE: THESE TAILWIND IMPORTS SHOULD NEVER BE DELETED */
@tailwind base;
@tailwind components;
@tailwind utilities;
/* DO NOT DELETE THESE TAILWIND IMPORTS, OTHERWISE THE STYLING WILL NOT RENDER AT ALL */
.highlight-new {
animation: highlight-and-fade 3s ease-out;
}
@keyframes highlight-and-fade {
0% {
background-color: #dcfce7; /* A light green color */
box-shadow: 0 0 12px 2px #86efac;
}
70% {
background-color: #dcfce7;
box-shadow: 0 0 12px 2px #86efac;
}
100% {
background-color: #ffffff; /* Fade back to white */
box-shadow: 0 4px 6px -1px rgba(0, 0, 0, 0.1), 0 2px 4px -1px rgba(0, 0, 0, 0.06);
}
}
```
[15 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]