# Project export: Halo AI This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2025 - Tagline: Halo AI: Real-time AI for Emergency Response.Transcribes, analyzes, and triages 911 calls with AI to help dispatchers act faster and save lives. - Devpost: https://devpost.com/software/halo-dispatch-ai - GitHub: https://github.com/Omer-Ozturk-SJSU/Halo-Dispatch - Video: https://www.youtube.com/embed/YvAF3TPba08?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Omer-Ozturk-Java (17 commits), Harsh Soni (8 commits), MuratAlkan06 (4 commits), Caleb Chan (3 commits) ## Devpost submission (written by the team) No Devpost description available. ## README (from the GitHub repository) # Halo Dispatch Halo Dispatch is a 911 call analysis system that uses AI to extract key safety concerns and urgency scores from emergency call transcripts. ## Project Structure - **halo-backend**: FastAPI backend server with AI processing endpoints - **halo-frontend**: Next.js frontend application - **orkes**: Orkes Conductor workflow agents and testing tools ## Orkes Conductor Integration This project uses [Orkes Conductor](https://orkes.io/) to orchestrate AI workflows: - **urgency_score_agent**: Rates the urgency of 911 call transcripts on a scale of 1-10 - **key_concerns_agent**: Extracts key safety concerns from transcripts (e.g., Bleeding, Domestic Violence) ### Setting Up Orkes 1. Create an account on [Orkes Conductor](https://orkes.io/) 2. Register the workflow definitions from `orkes/workflows/` 3. Configure your API token in `orkes/.env` 4. See `orkes/README.md` for detailed instructions ### Testing Orkes Workflows ```bash # Navigate to the orkes directory cd orkes # Run the trigger script python trigger_agents.py --workflow urgency_score_agent python trigger_agents.py --workflow key_concerns_agent # Test with custom transcript python trigger_agents.py --transcript "I need help. Someone is breaking into my house." # Test with sample transcripts python trigger_agents.py --test-file tests/test_transcripts.json ``` ## Backend Setup The backend uses FastAPI with OpenAI integration for AI analysis. ```bash # Navigate to the backend directory cd halo-backend # Install dependencies pip install -r requirements.txt # Set OpenAI API key export OPENAI_API_KEY="l4j4aac28607-4f06-11f0-a795-d685533af8e3" # Run the server uvicorn main:app --reload ``` ### API Endpoints - `GET /`: Health check endpoint - `POST /api/ai/urgency-score`: Get urgency score for transcript - `POST /api/ai/key-concerns`: Extract key concerns from transcript ## Frontend Setup The frontend is built with Next.js and includes an Orkes client integration. ```bash # Navigate to the frontend directory cd halo-frontend # Install dependencies pnpm install # Set up environment variables cp .env.example .env.local # Edit .env.local with your Orkes credentials # Run the development server pnpm dev ``` ## Development Workflow 1. Start the backend server 2. Start the frontend development server 3. Test Orkes workflows using the trigger script 4. Use the test API script to verify backend endpoints ## Testing ```bash # Test backend API endpoints cd orkes python tests/test_backend_api.py # Test with specific transcript python tests/test_backend_api.py --transcript-id test-emergency-1 # Test only urgency endpoint python tests/test_backend_api.py --endpoint urgency ``` ## Detected evidence (automated analysis) Indexed codebase: 104 recognized source files, 574 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Vercel (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (120 of 120) ``` .gitignore debug_token.py docs/prompt_examples.md get_jwt_token.py halo-backend/api/key_concerns.py halo-backend/api/test_vapi.py halo-backend/api/urgency_score.py halo-backend/api/vapi_webhook.py halo-backend/db/init_db.py halo-backend/db/migrate_add_external_call_id.py halo-backend/db/migrate_sqlite.py halo-backend/db/models.py halo-backend/ENV_SETUP.md halo-backend/main.py halo-backend/orkes/check_workflow.py halo-backend/orkes/debug_token.py halo-backend/orkes/get_jwt_token.py halo-backend/orkes/package.json halo-backend/orkes/prompts/concerns_prompt.md halo-backend/orkes/prompts/urgency_prompt.md halo-backend/orkes/README.md halo-backend/orkes/requirements.txt halo-backend/orkes/tests/test_backend_api.py halo-backend/orkes/tests/test_transcripts.json halo-backend/orkes/trigger_agents.py halo-backend/orkes/trigger_orkes.py halo-backend/orkes/workflows/key_concerns_agent.json halo-backend/orkes/workflows/urgency_score_agent.json halo-backend/requirements.txt halo-backend/services/ai_prompts.py halo-backend/services/socket_server.py halo-backend/setup_vapi_forwarding.py halo-backend/VAPI_SETUP_GUIDE.md halo-frontend/app/globals.css halo-frontend/app/layout.tsx halo-frontend/app/live-transcript/page.tsx halo-frontend/app/page.tsx halo-frontend/components.json halo-frontend/components/InteractiveMap.tsx halo-frontend/components/LiveTranscriptDashboard.tsx halo-frontend/components/theme-provider.tsx halo-frontend/components/ui/accordion.tsx halo-frontend/components/ui/alert-dialog.tsx halo-frontend/components/ui/alert.tsx halo-frontend/components/ui/aspect-ratio.tsx halo-frontend/components/ui/avatar.tsx halo-frontend/components/ui/badge.tsx halo-frontend/components/ui/breadcrumb.tsx halo-frontend/components/ui/button.tsx halo-frontend/components/ui/calendar.tsx halo-frontend/components/ui/card.tsx halo-frontend/components/ui/carousel.tsx halo-frontend/components/ui/chart.tsx halo-frontend/components/ui/checkbox.tsx halo-frontend/components/ui/collapsible.tsx halo-frontend/components/ui/command.tsx halo-frontend/components/ui/context-menu.tsx halo-frontend/components/ui/dialog.tsx halo-frontend/components/ui/drawer.tsx halo-frontend/components/ui/dropdown-menu.tsx halo-frontend/components/ui/form.tsx halo-frontend/components/ui/hover-card.tsx halo-frontend/components/ui/input-otp.tsx halo-frontend/components/ui/input.tsx halo-frontend/components/ui/label.tsx halo-frontend/components/ui/menubar.tsx halo-frontend/components/ui/navigation-menu.tsx halo-frontend/components/ui/pagination.tsx halo-frontend/components/ui/popover.tsx halo-frontend/components/ui/progress.tsx halo-frontend/components/ui/radio-group.tsx halo-frontend/components/ui/resizable.tsx halo-frontend/components/ui/scroll-area.tsx halo-frontend/components/ui/select.tsx halo-frontend/components/ui/separator.tsx halo-frontend/components/ui/sheet.tsx halo-frontend/components/ui/sidebar.tsx halo-frontend/components/ui/skeleton.tsx halo-frontend/components/ui/slider.tsx halo-frontend/components/ui/sonner.tsx halo-frontend/components/ui/switch.tsx halo-frontend/components/ui/table.tsx halo-frontend/components/ui/tabs.tsx halo-frontend/components/ui/textarea.tsx halo-frontend/components/ui/toast.tsx halo-frontend/components/ui/toaster.tsx halo-frontend/components/ui/toggle-group.tsx halo-frontend/components/ui/toggle.tsx halo-frontend/components/ui/tooltip.tsx halo-frontend/components/ui/use-mobile.tsx halo-frontend/components/ui/use-toast.ts halo-frontend/hooks/use-mobile.tsx halo-frontend/hooks/use-toast.ts halo-frontend/lib/orkes-example.ts halo-frontend/lib/orkes-service.ts halo-frontend/lib/utils.ts halo-frontend/MAP_SETUP.md halo-frontend/next-env.d.ts halo-frontend/next.config.mjs halo-frontend/ORKES_SETUP.md halo-frontend/package.json halo-frontend/postcss.config.mjs halo-frontend/styles/globals.css halo-frontend/tailwind.config.ts halo-frontend/tsconfig.json orkes/debug_token.py orkes/get_jwt_token.py orkes/package.json orkes/prompts/concerns_prompt.md orkes/prompts/urgency_prompt.md orkes/README.md orkes/requirements.txt orkes/tests/test_backend_api.py orkes/tests/test_transcripts.json orkes/trigger_agents.py orkes/workflows/key_concerns_agent.json orkes/workflows/urgency_score_agent.json README.md trigger_agents.py trigger_orkes.py ``` ### Dependencies - halo-backend/orkes/package.json: dotenv@^16.3.1 - halo-backend/orkes/requirements.txt: python-dotenv@==1.0.0, requests@==2.31.0 - halo-backend/requirements.txt: fastapi@==0.104.1, mysql-connector-python, openai@==1.3.0, pydantic@==2.4.2, python-dotenv@==1.0.0, sqlalchemy@==2.0.23, uvicorn@==0.24.0, websockets@==12.0 - halo-frontend/package.json: @hookform/resolvers@^3.9.1, @io-orkes/conductor-javascript@^2.1.4, @radix-ui/react-accordion@1.2.2, @radix-ui/react-alert-dialog@1.1.4, @radix-ui/react-aspect-ratio@1.1.1, @radix-ui/react-avatar@1.1.2, @radix-ui/react-checkbox@1.1.3, @radix-ui/react-collapsible@1.1.2, @radix-ui/react-context-menu@2.2.4, @radix-ui/react-dialog@1.1.4, @radix-ui/react-dropdown-menu@2.1.4, @radix-ui/react-hover-card@1.1.4, @radix-ui/react-label@2.1.1, @radix-ui/react-menubar@1.1.4, @radix-ui/react-navigation-menu@1.2.3, @radix-ui/react-popover@1.1.4, @radix-ui/react-progress@1.1.1, @radix-ui/react-radio-group@1.2.2, @radix-ui/react-scroll-area@1.2.2, @radix-ui/react-select@2.1.4, @radix-ui/react-separator@1.1.1, @radix-ui/react-slider@1.2.2, @radix-ui/react-slot@1.1.1, @radix-ui/react-switch@1.1.2, @radix-ui/react-tabs@1.1.2, @radix-ui/react-toast@1.2.4, @radix-ui/react-toggle@1.1.1, @radix-ui/react-toggle-group@1.1.1, @radix-ui/react-tooltip@1.1.6, @turf/turf@^7.2.0, @types/leaflet@^1.9.18, @types/node@^22, @types/react@^19, @types/react-dom@^19, @types/turf@^3.5.32, autoprefixer@^10.4.20, class-variance-authority@^0.7.1, clsx@^2.1.1, cmdk@1.0.4, date-fns@4.1.0, embla-carousel-react@8.5.1, input-otp@1.4.1, leaflet@^1.9.4, lucide-react@^0.454.0, next@15.2.4, next-themes@^0.4.4, postcss@^8.5, react@^19, react-day-picker@8.10.1, react-dom@^19, react-hook-form@^7.54.1, react-leaflet@^5.0.0, react-resizable-panels@^2.1.7, recharts@2.15.0, sonner@^1.7.1, tailwind-merge@^2.5.5, tailwindcss@^3.4.17, tailwindcss-animate@^1.0.7, typescript@^5, vaul@^0.9.6, zod@^3.24.1 - orkes/package.json: dotenv@^16.3.1 - orkes/requirements.txt: python-dotenv@==1.0.0, requests@==2.31.0 ### Recent commits (newest first) - Merge branch 'main' of https://github.com/Omer-Ozturk-SJSU/Halo-Dispatch - changes orkes - Add Orkes workflow trigger and status check scripts - Last change - Resolve merge conflicts and move orkes directory into halo-backend - delete - Merge branch 'main' of https://github.com/Omer-Ozturk-SJSU/Halo-Dispatch - save to db - yftyfty - Merge remote changes and apply stashed updates - chore: add comprehensive .gitignore and package-lock.json - feat: implement VAPI call forwarding with live transcription - Merge branch 'main' of https://github.com/Omer-Ozturk-SJSU/Halo-Dispatch - Add temp 2.txt file - Other endpoints - CRUD - data base initialized + models - models. - vapi API - main ## Key source files (fetched from GitHub, selected and truncated for size) ### docs/prompt_examples.md ```markdown # Example GPT Prompts for AI Agents ## Urgency Prompt Based on this 911 call, assign an urgency from 1 (low) to 10 (life-threatening). Return only a number. ## Key Concerns Prompt Identify and return only key safety concerns: ["Domestic Violence", "Bleeding", "Head Injury", "Perp Present", "Mental Health", "Unknown Location"]. ``` ### halo-backend/ENV_SETUP.md ```markdown # Environment Setup Guide To properly configure the Orkes integration, you need to set up your `.env` file in the `halo-backend` directory. ## Create the .env file Create a file named `.env` in the `halo-backend` directory with the following content: ``` # Orkes Configuration ORKES_SERVER_URL=https://developer.orkescloud.com ORKES_KEY_ID=your-key-id-here ORKES_KEY_SECRET=your-key-secret-here # OpenAI Configuration OPENAI_API_KEY=your-openai-api-key # Database Configuration DATABASE_URL=sqlite:///./halo.db ``` ## Getting Orkes Credentials To get your Orkes credentials: 1. Log in to your Orkes Conductor dashboard at https://developer.orkescloud.com 2. Navigate to **Access Control** > **Applications** 3. Select your application or create a new one 4. In the **Access Keys** section, select **+ Create access key** 5. This will generate a unique Key ID and Key Secret 6. Copy these values to your `.env` file ## Testing the Orkes Integration After setting up your `.env` file, you can test the Orkes integration by running: ```bash cd halo-backend/orkes python trigger_agents.py ``` This will attempt to trigger a workflow in Orkes using your credentials. ``` ### orkes/requirements.txt ``` requests==2.31.0 python-dotenv==1.0.0 ``` ### halo-backend/requirements.txt ``` # Halo Dispatch API requirements fastapi==0.104.1 uvicorn==0.24.0 pydantic==2.4.2 openai==1.3.0 python-dotenv==1.0.0 websockets==12.0 sqlalchemy==2.0.23 mysql-connector-python ``` ### orkes/package.json ``` { "name": "orkes", "version": "1.0.0", "description": "Orkes Conductor workflow agents for Halo Dispatch", "main": "trigger_agents.py", "scripts": { "start": "python trigger_agents.py", "test": "python tests/test_backend_api.py", "dev": "python trigger_agents.py", "test:urgency": "python tests/test_backend_api.py --endpoint urgency", "test:concerns": "python tests/test_backend_api.py --endpoint concerns" }, "dependencies": { "dotenv": "^16.3.1" } } ``` ### halo-frontend/package.json ``` { "name": "my-v0-project", "version": "0.1.0", "private": true, "scripts": { "build": "next build", "dev": "next dev", "lint": "next lint", "start": "next start" }, "dependencies": { "@hookform/resolvers": "^3.9.1", "@io-orkes/conductor-javascript": "^2.1.4", "@radix-ui/react-accordion": "1.2.2", "@radix-ui/react-alert-dialog": "1.1.4", "@radix-ui/react-aspect-ratio": "1.1.1", "@radix-ui/react-avatar": "1.1.2", "@radix-ui/react-checkbox": "1.1.3", "@radix-ui/react-collapsible": "1.1.2", "@radix-ui/react-context-menu": "2.2.4", "@radix-ui/react-dialog": "1.1.4", "@radix-ui/react-dropdown-menu": "2.1.4", "@radix-ui/react-hover-card": "1.1.4", "@radix-ui/react-label": "2.1.1", "@radix-ui/react-menubar": "1.1.4", "@radix-ui/react-navigation-menu": "1.2.3", "@radix-ui/react-popover": "1.1.4", "@radix-ui/react-progress": "1.1.1", "@radix-ui/react-radio-group": "1.2.2", "@radix-ui/react-scroll-area": "1.2.2", "@radix-ui/react-select": "2.1.4", "@radix-ui/react-separator": "1.1.1", "@radix-ui/react-slider": "1.2.2", "@radix-ui/react-slot": "1.1.1", "@radix-ui/react-switch": "1.1.2", "@radix-ui/react-tabs": "1.1.2", "@radix-ui/react-toast": "1.2.4", "@radix-ui/react-toggle": "1.1.1", "@radix-ui/react-toggle-group": "1.1.1", "@radix-ui/react-tooltip": "1.1.6", "@turf/turf": "^7.2.0", "@types/leaflet": "^1.9.18", "@types/turf": "^3.5.32", "autoprefixer": "^10.4.20", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "cmdk": "1.0.4", "date-fns": "4.1.0", "embla-carousel-react": "8.5.1", "input-otp": "1.4.1", "leaflet": "^1.9.4", "lucide-react": "^0.454.0", "next": "15.2.4", "next-themes": "^0.4.4", "react": "^19", "react-day-picker": "8.10.1", "react-dom": "^19", "react-hook-form": "^7.54.1", "react-leaflet": "^5.0.0", "react-resizable-panels": "^2.1.7", "recharts": "2.15.0", "sonner": "^1.7.1", "tailwind-merge": "^2.5.5", "tailwindcss-animate": "^1.0.7", "vaul": "^0.9.6", "zod": "^3.24.1" }, "devDependencies": { "@types/node": "^22", "@types/react": "^19", "@types/react-dom": "^19", "postcss": "^8.5", "tailwindcss": "^3.4.17", "typescript": "^5" } } ``` ### halo-backend/orkes/requirements.txt ``` requests==2.31.0 python-dotenv==1.0.0 ``` ### halo-backend/orkes/package.json ``` { "name": "orkes", "version": "1.0.0", "description": "Orkes Conductor workflow agents for Halo Dispatch", "main": "trigger_agents.py", "scripts": { "start": "python trigger_agents.py", "test": "python tests/test_backend_api.py", "dev": "python trigger_agents.py", "test:urgency": "python tests/test_backend_api.py --endpoint urgency", "test:concerns": "python tests/test_backend_api.py --endpoint concerns" }, "dependencies": { "dotenv": "^16.3.1" } } ``` ### halo-backend/main.py ```python from fastapi import FastAPI, Depends, HTTPException from fastapi.middleware.cors import CORSMiddleware from dotenv import load_dotenv import os from sqlalchemy.orm import Session from typing import List, Optional from datetime import datetime, timedelta # Load environment variables from .env load_dotenv() app = FastAPI(title="Halo Dispatch API") # Configure CORS app.add_middleware( CORSMiddleware, allow_origins=["*"], # In production, replace with specific origins allow_credentials=True, allow_methods=["*"], allow_headers=["*"], ) # Import and include all routers from api import vapi_webhook, test_vapi from api.urgency_score import router as urgency_router from api.key_concerns import router as concerns_router app.include_router(vapi_webhook.router) app.include_router(test_vapi.router) app.include_router(urgency_router) app.include_router(concerns_router) # Import the database models and engine from db.models import Base, engine, User, get_db from db.models import Transcript, AIInsight, Unit, Call from db.models import SessionLocal from pydantic import BaseModel class UserCreate(BaseModel): name: str address: str = None medical_conditions: str = None allergies: str = None emergency_contact: str = None class UserRead(BaseModel): id: int name: str address: str = None medical_conditions: str = None allergies: str = None emergency_contact: str = None class Config: from_attributes = True @app.post("/users/", response_model=UserRead) def create_user(user: UserCreate, db: Session = Depends(get_db)): db_user = User(**user.dict()) db.add(db_user) db.commit() db.refresh(db_user) return db_user @app.get("/users/", response_model=List[UserRead]) def read_users(skip: int = 0, limit: int = 10, db: Session = Depends(get_db)): users = db.query(User).offset(skip).limit(limit).all() return users @app.get("/users/{user_id}", response_model=UserRead) def read_user(user_id: int, db: Session = Depends(get_db)): user = db.query(User).filter(User.id == user_id).first() if user is None: raise HTTPException(status_code=404, detail="User not found") return user @app.put("/users/{user_id}", response_model=UserRead) def update_user(user_id: int, user: UserCreate, db: Session = Depends(get_db)): db_user = db.query(User).filter(User.id == user_id).first() if db_user is None: raise HTTPException(status_code=404, detail="User not found") for key, value in user.dict().items(): setattr(db_user, key, value) db.commit() db.refresh(db_user) return db_user @app.delete("/users/{user_id}") def delete_user(user_id: int, db: Session = Depends(get_db)): db_user = db.query(User).filter(User.id == user_id).first() if db_user is None: raise HTTPException(status_code=404, detail="User not found") db.delete(db_user) db.commit() return {"detail": "User deleted"} @app.get("/") def root(): return {"status": "API running", "service": "Halo Dispatch API"} class TranscriptRead(BaseModel): id: int call_id: int speaker: str timestamp: datetime text: str class Config: from_attributes = True @app.get("/calls/{call_id}/transcripts", response_model=List[TranscriptRead]) def get_transcripts(call_id: int, db: Session = Depends(get_db)): transcripts = db.query(Transcript).filter(Transcript.call_id == call_id).order_by(Transcript.timestamp).all() return transcripts class AIInsightRead(BaseModel): id: int call_id: int concern_tags: str urgency_score: int class Config: from_attributes = True @app.get("/calls/{call_id}/ai-insights", response_model=AIInsightRead) def get_ai_insight(call_id: int, db: Session = Depends(get_db)): insight = db.query(AIInsight).filter(AIInsight.call_id == call_id).order_by(AIInsight.id.desc()).first() if insight is None: raise HTTPException(status_code=404, detail="AI insight not found") return insight class UnitRead(BaseModel): id: int call_id: int status: str eta: datetime = None location: str = None class Config: from_attributes = True @app.get("/calls/{call_id}/units", response_model=List[UnitRead]) def get_units(call_id: int, db: Session = Depends(get_db)): units = db.query(Unit).filter(Unit.call_id == call_id).all() return units class CallRead(BaseModel): id: int user_id: int timestamp: datetime current_score: Optional[int] = None status: str external_call_id: Optional[str] = None # Add the missing field class Config: from_attributes = True @app.get("/calls/", response_model=List[CallRead]) def read_calls(skip: int = 0, limit: int = 10, db: Session = Depends(get_db)): calls = db.query(Call).offset(skip).limit(limit).all() return calls @app.get("/calls/{call_id}", response_model=CallRead) def read_call(call_id: int, db: Session = Depends(get_db)): call = db.query(Call).filter(Call.id == call_id).first() if call is None: raise HTTPException(status_code=404, detail="Call not found") return call if __name__ == "__main__": print("Creating tables if they do not exist...") Base.metadata.create_all(bind=engine) print("Tables created.") # --- SEED DATA --- db = SessionLocal() # Create user user = User( name="Sarah Johnson", address="2495 Bancroft Way, Berkeley, CA", medical_conditions="Anxiety disorder; Previous concussion", allergies="Penicillin", emergency_contact="Linda Johnson (510) 555-6543" ) db.add(user) db.commit() db.refresh(user) # Create call call = Call( user_id=user.id, timestamp=datetime.now(), current_score=8, status="active" ) db.add(call) db.commit() db.refresh(call) # Create transcripts transcript_data = [ ("CALLER", "911, please help me...", 0), [truncated — 1124 more characters] ``` ### halo-frontend/app/layout.tsx ```typescript import type { Metadata } from 'next' import './globals.css' export const metadata: Metadata = { title: 'v0 App', description: 'Created with v0', generator: 'v0.dev', } export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode }>) { return ( {children} ) } ``` [95 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]