# Project export: Echocare This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: 3.4 million individuals are food insecure in SF. 150,000 tonnes of food are wasted every year in SF. Echocare leverages A.I. and advanced tools to kill these 2 problems with 1 stone. - Devpost: https://devpost.com/software/echocare-i6csrv - GitHub: https://github.com/ppinkfreudd/echocare - Video: https://www.youtube.com/embed/CMhPyazI5N4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Vapi: Show us your Voice AI) - Team: 2 GitHub contributor(s) — SujashB (17 commits), Rishit Das (13 commits) ## Devpost submission (written by the team) ### Overview Echocare TABLE NUMBER 100 ### Inspiration In the U.S. and S.F., homelessness continues to be a persistent issue, affecting more than 650,000 individuals in 2023 alone. Despite the lack of stable housing, many people experiencing homelessness own mobile phones. Research indicates that as many as 94% of homeless individuals have access to a cellphone, with around 70% having smartphones. This is crucial because phones act as a lifeline for communication, health services, and access to support networks. Phones help users connect with essential services like medical care, job opportunities, and safety alerts. Unfortunately, barriers such as maintaining a charge or affording phone plans remain common issues for this population. In addition, U.S. faces a huge problem of food wastage. 150,000 tonnes of food are wasted each year JUST in SF. With Echocare, we aim to kill these 2 problems with 1 stone. We leverage S.O.T.A Artificial Intelligence to provide essential services to homeless and people suffering from food insecurity in a more accessible and intuitive manner. Additionally, we allow restaurants to donate and keep track of leftover food using an inventory tracker which can be used to feed the homeless people. ### What it does Echocare is an intuitive platform designed to assist users in locating and connecting with nearby care services, such as medical centers, pharmacies, or home care providers. The application uses voice input, interactive maps, and real-time search functionalities to help users find the services they need quickly and seamlessly. With user-friendly navigation and smart recommendations, Echocare empowers people to get help with minimal effort. In addition, it offers a separate platform for restaurants to keep track of leftover food and donate them at the end of each business day to homeless people. ### How we built it Next.js for server-side rendering and static generation. React to build interactive and modular UI components for the front end. TypeScript for type safety and a robust backend. Tailwind CSS for rapid, utility-first styling of the application. Framer Motion for smooth and declarative animations. GSAP (GreenSock Animation Platform) for high-performance animations. (Used for Echo) Three.js for creating 3D graphics in the browser, adding depth and interactivity. (Echo has 3d graphics built in it) Vercel Postgres to communicate with Neon DB (Serverless Postgres) Neon Database a serverless PostgreSQL option, for robust backend storage to store the donated food items of the restaurants. Uses minimal compute and is good for developing low-latency applications Clerk to implement secure and seamless user authentication for managers of the restaurants who want to donate food. Google Maps API to power the mapping functionality and location services (This was embedded with Echo to provide precise directions). Google Places API for autocomplete suggestions and retrieving detailed place information. Ant Design and Aceternity UI for building the forms in the food donation page and having a clean look for the landing page (inspired by the multicolored and vibrant lights of SF in the night) Axios for making API requests to external services easily (Google Maps and Places API) Lucide React for all of the icons used in the application. Vapi.ai For creating the one and only assistant Google Gemini Flash 1.5 to potentially assist with generating user-facing responses. Groq 3.1 70b versatile (fine-tuned) to assist Vapi.ai with insights. Cartesia to provide hyper-realistic service for users. Deepgram for encoding ### Challenges we ran into We found it very difficult to transcribe the conversations between the user and Vapi.ai echo agent. Rishit had to code for 16 hours straight to get it working. We also found the Google Gemini Integration to be hard because the multimodal functionality wasn't easy to implement. Especially with Typescript which isn't well documented as compared to Python. Finally, stitching the backend and frontend together in the food donation page also took a lot of time to carry out. ### Accomplishments we're proud of Getting 4 sponsored tools to be seamlessly integrated in the application Using a grand total of 18 tools to build Echocare from the ground up Finishing the entire product 10 hours before the deadline Creating a product which can truly be used to help burdened communities thrive Having a lot of fun and enjoying the process of building Echocare! ### What we learned Typescript - We have never used it to build a project before and through this hackathon, we gained understanding of how we can use it to build cohesive applications Vapi.AI - Using the dashboard and integrating custom APIs into Vapi was a tricky operation, but we toughed it out and made it work. NeonDB - We used this DB and learned basic SQL queries to insert and get data from the DB which we used to setup the donation page. Google Maps/Places API - Although they were relatively easy to implement, some time had to spent to initialize them. Groq, Cartesia - They were definitely tricky to implement with Vapi's dashboard. Gemini Integration with TS - Although it was < 100 lines of code, we kept running into errors. Turned out that Gemini Pro Vision was deprecated and we had to use Gemini 1.5 Flash instead ;( ### What's next We would like to train our VAPI RAG with more datasets from other cities in the United States. Also, we would like to improve the responsive dimensions of our website. ## README (from the GitHub repository) ### Inspiration In the U.S. and S.F., homelessness continues to be a persistent issue, affecting more than 650,000 individuals in 2023 alone. Despite the lack of stable housing, many people experiencing homelessness own mobile phones. Research indicates that as many as **94% of homeless individuals** have access to a cellphone, with around **70%** having smartphones. This is crucial because phones act as a lifeline for communication, health services, and access to support networks. Phones help users connect with essential services like medical care, job opportunities, and safety alerts. Unfortunately, barriers such as maintaining a charge or affording phone plans remain common issues for this population. In addition, U.S. faces a huge problem of food wastage. 150,000 tonnes of food are wasted each year JUST in SF. With Echocare, we aim to kill these 2 problems with 1 stone. We leverage S.O.T.A Artificial Intelligence to provide essential services to homeless people in a more accessible and intuitive manner. Additionally, we allow restaurants to donate and keep track of leftover food using an inventory tracker which can be used to feed the homeless people. ### What it Does Echocare is an intuitive platform designed to assist users in locating and connecting with nearby care services, such as medical centers, pharmacies, or home care providers. The application uses voice input, interactive maps, and real-time search functionalities to help users find the services they need quickly and seamlessly. With user-friendly navigation and smart recommendations, Echocare empowers people to get help with minimal effort. In addition, it offers a separate platform for restaurants to keep track of leftover food and donate them at the end of each business day to homeless people. ### How We Built It - **Next.js** for server-side rendering and static generation. - **React** to build interactive and modular UI components for the front end. - **TypeScript** for type safety and a robust backend. - **Tailwind CSS** for rapid, utility-first styling of the application. - **Framer Motion** for smooth and declarative animations. - **GSAP (GreenSock Animation Platform)** for high-performance animations. (Used for Echo) - **Three.js** for creating 3D graphics in the browser, adding depth and interactivity. (Echo has 3d graphics built in it) - **Vercel Postgres** to communicate with Neon DB (Serverless Postgres) - **Neon Database** a serverless PostgreSQL option, for robust backend storage to store the donated food items of the restaurants. Uses minimal compute and is good for developing low-latency applications - **Clerk** to implement secure and seamless user authentication for managers of the restaurants who want to donate food. - **Google Maps API** to power the mapping functionality and location services (This was embedded with Echo to provide precise directions). - **Google Places API** for autocomplete suggestions and retrieving detailed place information. - **Ant Design, Aceternity UI** for building the forms in the food donation page and having a clean look for the landing page (inspired by the multicolored and vibrant lights of SF in the night) - **Axios** for making API requests to external services easily (Google Maps and Places API) - **Lucide React** for all of the icons used in the application. - **Vapi.ai** For creating the one and only assistant - **Google Gemini Flash 1.5** to potentially assist with generating user-facing responses. - **Groq 3.1 70b versatile (fine-tuned)** to assist Vapi.ai with insights. - **Cartesia** to provide hyper-realistic service for users. ### Challenges We Ran Into We found it very difficult to transcribe the conversations between the user and Vapi.ai echo agent. Rishit had to code for 16 hours straight to get it working. We also found the Google Gemini Integration to be hard because the multimodal functionality wasn't easy to implement. Especially with Typescript which isn't well documented as compared to Python. Finally, stitching the backend and frontend together in the food donation page also took a lot of time to carry out. ### Accomplishments We Are Proud Of - Getting 4 sponsored tools to be seamlessly integrated in the application - Using a grand total of 18 tools to build Echocare from the ground up - Finishing the entire product 10 hours before the deadline - Creating a product which can truly be used to help burdened communities thrive - Having a lot of fun and enjoying the process of building Echocare! ### What We Learned - Typescript - We have never used it to build a project before and through this hackathon, we gained understanding of how we can use it to build cohesive applications - Vapi.AI - Using the dashboard and integrating custom APIs into Vapi was a tricky operation, but we toughed it out and made it work. - NeonDB - We used this DB and learned basic SQL queries to insert and get data from the DB which we used to setup the donation page. - Google Maps/Places API - Although they were relatively easy to implement, some time had to spent to initialize them. - Groq, Cartesia - They were definitely tricky to implement with Vapi's dashboard. - Gemini Integration with TS - Although it was < 100 lines of code, we kept running into errors. Turned out that Gemini Pro Vision was disabled and we had to use Gemini 1.5 Flash instead ;( ### Contributors - **Rishit Das** - **Sujash Barman** ## Detected evidence (automated analysis) Indexed codebase: 23 recognized source files, 70 KB. - CSS (language) — detected in the code - Google Gemini (technology) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (30 of 30) ``` .env echocare/.gitignore echocare/app/actions.ts echocare/app/businesspage_side/page.tsx echocare/app/components/form.tsx echocare/app/components/gemini-food-vision.ts echocare/app/components/gemini-food-vision.tsx echocare/app/components/glob.tsx echocare/app/components/MicButton.tsx echocare/app/components/Transcriber.tsx echocare/app/components/ui/background-gradient.tsx echocare/app/components/ui/moving-border.tsx echocare/app/components/ui/RainbowButton.tsx echocare/app/components/ui/Ripple.tsx echocare/app/globals.css echocare/app/hooks/use-vapi.ts echocare/app/landing_page/landing_page.tsx echocare/app/layout.tsx echocare/app/page.tsx echocare/app/providers.tsx echocare/lib/utils.ts echocare/next.config.mjs echocare/package.json echocare/postcss.config.mjs echocare/README.md echocare/tailwind.config.ts echocare/tsconfig.json next.config.js README.md tsconfig.json ``` ### Dependencies - echocare/package.json: @clerk/nextjs@^5.7.5, @google/generative-ai@^0.21.0, @googlemaps/js-api-loader@^1.16.8, @neondatabase/serverless@^0.10.1, @radix-ui/react-avatar@^1.1.1, @react-google-maps/api@^2.20.3, @types/node@^20, @types/react@^18, @types/react-dom@^18, @vapi-ai/web@^2.1.4, @vercel/postgres@^0.10.0, antd@^5.21.4, axios@^1.7.7, clsx@^2.1.1, cors@^2.8.5, dotenv@^16.4.5, framer-motion@^11.11.9, google@^2.1.0, gsap@^3.12.5, lib-utils-ts@^0.0.0-test, lucide-react@^0.453.0, next@14.2.15, postcss@^8, react@^18, react-dom@^18, tailwind-merge@^2.5.4, tailwindcss@^3.4.1, three@^0.169.0, typescript@^5, utils@^0.3.1 ### Recent commits (newest first) - Delete echocare/.env - Update .env - changes - fixed gemini output - Merge branch 'main' of https://github.com/ppinkfreudd/echocare - Changes to fix landing page - Update README.md - CalHacks ready - Update README.md - Merge pull request #5 from ppinkfreudd/frontend-redesign - finished frontend redesign - Merge branch 'main' of https://github.com/ppinkfreudd/echocare - Save local changes before pulling - Merge pull request #4 from ppinkfreudd/food-security - Merge branch 'main' into food-security - Finished Gemini Pro Shit - changes to vapi - fixed stuff - Merge branch 'main' of https://github.com/ppinkfreudd/echocare - Made some basic changes. ## Key source files (fetched from GitHub, selected and truncated for size) ### echocare/package.json ``` { "name": "echocare", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@clerk/nextjs": "^5.7.5", "@google/generative-ai": "^0.21.0", "@googlemaps/js-api-loader": "^1.16.8", "@neondatabase/serverless": "^0.10.1", "@radix-ui/react-avatar": "^1.1.1", "@react-google-maps/api": "^2.20.3", "@vapi-ai/web": "^2.1.4", "@vercel/postgres": "^0.10.0", "antd": "^5.21.4", "axios": "^1.7.7", "clsx": "^2.1.1", "cors": "^2.8.5", "dotenv": "^16.4.5", "framer-motion": "^11.11.9", "google": "^2.1.0", "gsap": "^3.12.5", "lib-utils-ts": "^0.0.0-test", "lucide-react": "^0.453.0", "next": "14.2.15", "react": "^18", "react-dom": "^18", "tailwind-merge": "^2.5.4", "three": "^0.169.0", "utils": "^0.3.1" }, "devDependencies": { "@types/node": "^20", "@types/react": "^18", "@types/react-dom": "^18", "postcss": "^8", "tailwindcss": "^3.4.1", "typescript": "^5" }, "browser": { "fs": false, "path": false, "os": false } } ``` ### echocare/app/page.tsx ```typescript import LandingPage from '../app/landing_page/landing_page' import BusinessPage from '../app/businesspage_side/business_page'; export default function Home() { return } ``` ### echocare/app/layout.tsx ```typescript import type { Metadata } from "next"; import localFont from "next/font/local"; import "./globals.css"; import { ClerkProvider } from '@clerk/nextjs' const geistSans = localFont({ src: "./fonts/GeistVF.woff", variable: "--font-geist-sans", weight: "100 900", }); const geistMono = localFont({ src: "./fonts/GeistMonoVF.woff", variable: "--font-geist-mono", weight: "100 900", }); export const metadata: Metadata = { title: "Create Next App", description: "Generated by create next app", }; export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return ( {children} ); } ``` ### echocare/app/businesspage_side/page.tsx ```typescript "use client" import React, { useState, useEffect } from 'react'; import { Button as MovingBorderButton } from '../components/ui/moving-border'; import FoodDonationForm from '../components/form'; import { Modal, Card, Row, Col, Image, message } from 'antd'; import { addDonation, getDonations } from '../actions'; import { useUser } from "@clerk/nextjs"; interface DonationItem { id: number; restaurant_name: string; food_name: string; food_category: string; quantity: number; description: string; photo_url?: string; } const Page: React.FC = () => { const { user } = useUser(); const [isModalVisible, setIsModalVisible] = useState(false); const [donations, setDonations] = useState([]); useEffect(() => { fetchDonations(); }, []); const fetchDonations = async () => { const result = await getDonations(); if (result.success && Array.isArray(result.donations)) { setDonations(result.donations as DonationItem[]); } else { message.error(`Failed to fetch donations: ${result.error || 'Unknown error'}`); } }; const showModal = () => { setIsModalVisible(true); }; const handleCancel = () => { setIsModalVisible(false); }; const handleFormSubmit = async (values: any) => { try { console.log('Submitting form with values:', values); const result = await addDonation(values); console.log('Received result from addDonation:', result); if (result.success) { message.success('Donation added successfully!'); setIsModalVisible(false); fetchDonations(); // Refresh the donations list } else { console.error('Server returned error:', result.error); message.error(`Failed to add donation: ${result.error}`); } } catch (error) { console.error('Client-side error:', error); message.error(`An unexpected error occurred: ${error instanceof Error ? error.message : String(error)}`); } }; return (

Welcome, {user ? user.fullName || user.username || 'User' : 'User'}

Ready to donate some food? Click the button below to get started!

Add Product
{donations.map((donation) => ( {donation.food_name}
)} className="bg-slate-800 text-white border-slate-700" > {donation.food_name}} description={

Category: {donation.food_category}

Quantity: {donation.quantity}

Description: {donation.description}

} /> ))}
); }; export default Page; ``` ### echocare/app/landing_page/landing_page.tsx ```typescript "use client"; import React, { useState, useRef, useEffect } from 'react'; import AbstractBall from '../components/glob'; import Transcriber from '../components/Transcriber'; import useVapi from '../hooks/use-vapi'; import MicButton from '../components/MicButton'; import { MicIcon, PhoneOff } from 'lucide-react'; import { motion } from 'framer-motion'; import { useRouter } from 'next/navigation'; import { useAuth, SignInButton } from "@clerk/nextjs"; import { identifyFoodSafety } from '../components/gemini-food-vision'; import Ripple from '../components/ui/Ripple'; import RainbowButton from '../components/ui/RainbowButton'; import { GoogleMap, LoadScriptNext, Marker, Autocomplete } from '@react-google-maps/api'; const mapContainerStyle = { width: '300px', height: '300px', }; const darkModeStyle = [ { "elementType": "geometry", "stylers": [{ "color": "#212121" }] }, { "elementType": "labels.icon", "stylers": [{ "visibility": "off" }] }, { "elementType": "labels.text.fill", "stylers": [{ "color": "#757575" }] }, { "elementType": "labels.text.stroke", "stylers": [{ "color": "#212121" }] }, { "featureType": "administrative", "elementType": "geometry", "stylers": [{ "color": "#757575" }] }, { "featureType": "administrative.country", "elementType": "labels.text.fill", "stylers": [{ "color": "#9e9e9e" }] }, { "featureType": "administrative.land_parcel", "stylers": [{ "visibility": "off" }] }, { "featureType": "administrative.locality", "elementType": "labels.text.fill", "stylers": [{ "color": "#bdbdbd" }] }, { "featureType": "poi", "elementType": "labels.text.fill", "stylers": [{ "color": "#757575" }] }, { "featureType": "road", "elementType": "geometry.fill", "stylers": [{ "color": "#2c2c2c" }] }, { "featureType": "road", "elementType": "labels.text.fill", "stylers": [{ "color": "#8a8a8a" }] }, { "featureType": "road.arterial", "elementType": "geometry", "stylers": [{ "color": "#373737" }] }, { "featureType": "road.highway", "elementType": "geometry", "stylers": [{ "color": "#3c3c3c" }] }, { "featureType": "road.highway.controlled_access", "elementType": "geometry", "stylers": [{ "color": "#4e4e4e" }] }, { "featureType": "road.local", "elementType": "labels.text.fill", "stylers": [{ "color": "#616161" }] }, { "featureType": "transit", "elementType": "labels.text.fill", "stylers": [{ "color": "#757575" }] }, { "featureType": "water", "elementType": "geometry", "stylers": [{ "color": "#000000" }] }, { "featureType": "water", "elementType": "labels.text.fill", "stylers": [{ "color": "#3d3d3d" }] } ]; const LandingPage: React.FC = () => { const { isLoaded, isSignedIn } = useAuth(); const router = useRouter(); const { volumeLevel, isSessionActive, conversation, toggleCall } = useVapi(); const fileInputRef = useRef(null); const [foodSafetyResult, setFoodSafetyResult] = useState(null); const [config, setConfig] = useState({ perlinTime: 50.0, perlinDNoise: 1.0, chromaRGBr: 7.5, chromaRGBg: 5, chromaRGBb: 7, chromaRGBn: 0, chromaRGBm: 1.0, sphereWireframe: false, spherePoints: false, spherePsize: 0.1, cameraSpeedY: 0.0, cameraSpeedX: 0.0, cameraZoom: 175, cameraGuide: false, perlinMorph: 5.5, }); const [center, setCenter] = useState({ lat: 0, lng: 0 }); const [destination, setDestination] = useState(null); const autocompleteRef = useRef(null); useEffect(() => { const getUserLocation = () => { if (navigator.geolocation) { navigator.geolocation.getCurrentPosition((position) => { setCenter({ lat: position.coords.latitude, lng: position.coords.longitude, }); }, (error) => { console.error("Error getting location: ", error); }); } else { console.error("Geolocation is not supported by this browser."); } }; getUserLocation(); }, []); // Ensure any client-side only logic is wrapped in useEffect useEffect(() => { if (isSessionActive) { setConfig(prevConfig => ({ ...prevConfig, perlinTime: 100.0, perlinMorph: 25.0, })); } else { setConfig(prevConfig => ({ ...prevConfig, perlinTime: 5.0, perlinMorph: 0, })); } }, [isSessionActive]); const handleDonateClick = () => { if (isSignedIn) { router.push('/businesspage_side'); } }; const handleIdentifyFoodSafety = () => { fileInputRef.current?.click(); }; const handleFileChange = async (event: React.ChangeEvent) => { const file = event.target.files?.[0]; if (file) { try { setFoodSafetyResult("Analyzing image..."); const result = await identifyFoodSafety(file); setFoodSafetyResult(result); } catch (error) { console.error('Error identifying food safety:', error); setFoodSafetyResult('An error occurred while analyzing the image.'); } } }; const onLoad = (autocomplete: google.maps.places.Autocomplete) => { autocompleteRef.current = autocomplete; }; const onPlaceChanged = () => { if (autocompleteRef.current) { const place = autocompleteRef.current.getPlace(); if (place && place.geometry && place.geometry.location) { setDestination({ lat: place.geometry.location.lat(), lng: place.geometry.location.lng(), }); } else { console.error("Selected place does not have geometry information."); } } }; return (
[`--${key}`, val]) ); addBase({ ":root": newVars, }); } ``` ### echocare/lib/utils.ts ```typescript import { ClassValue, clsx } from "clsx"; import { twMerge } from "tailwind-merge"; export function cn(...inputs: ClassValue[]) { return twMerge(clsx(inputs)); } ``` ### echocare/app/providers.tsx ```typescript 'use client'; import { ClerkProvider } from '@clerk/nextjs'; export function Providers({ children }: { children: React.ReactNode }) { return {children}; } ``` ### echocare/app/globals.css ```css @tailwind base; @tailwind components; @tailwind utilities; :root { --background: #0a0a0a; --foreground: #ededed; --color-1: 0 100% 63%; --color-2: 270 100% 63%; --color-3: 210 100% 63%; --color-4: 195 100% 63%; --color-5: 90 100% 63%; } @media (prefers-color-scheme: dark) { :root { --background: #0a0a0a; --foreground: #ededed; } } body { color: var(--foreground); background: var(--background); font-family: Arial, Helvetica, sans-serif; } @layer utilities { .text-balance { text-wrap: balance; } } ``` [12 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]