# Project export: HawkWatch This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2025 - Tagline: Transform any camera into a smart security system with HawkWatch. Our AI VLM instantly detects and alerts security to potential threats, giving businesses of all sizes a peace of mind. - Devpost: https://devpost.com/software/hawkwatch - GitHub: https://github.com/Grace-Shao/Treehacks2025 - Video: https://www.youtube.com/embed/h2G_Z_NtcXk?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (TreeHacks Grand Prize | 1st Place ($11k Cash)) - Team: 4 GitHub contributor(s) — Alex (36 commits), Darryl Tanzil (19 commits), Grace-Shao (18 commits), nf-projects (9 commits) ## Devpost submission (written by the team) ### Inspiration In an era where security cameras are everywhere but meaningful surveillance is scarce, we saw an opportunity to transform passive recording systems into intelligent security guardians. Our inspiration came from real-world incidents where crucial moments were missed despite having camera coverage, and the overwhelming challenge security personnel face in monitoring multiple video feeds simultaneously. We wanted to create a solution that doesn't just record but understands, analyzes, and acts, whether it's for local businesses like grocery markets to bigger organizations like hospitals and shopping malls. ### What it does HawkWatch is an intelligent video surveillance platform that detects crime, suspicious activities and life threatening events such as fainting and choking and sends phone alerts to alert security of the issue. Our intelligent model generates time-stamped incident reports with video evidence. It has 4 main features: Real-time detection of dangerous activity is done by sending audio, video, and Tensorflow's body position data to Google's Gemini Visual Language Model, and sending email notification if needed An upload feature allows existing mp4 files to be analyzed A library of saved livestream footage and mp4 uploads, with detailed security analysis complete with timeline and information which is saved with each entry Statistics page which offers an AI summary, chart analysis, and the option to export situation data to CSV. Additional features Sends instant alerts to security through email/phone notifications set Provides an intuitive dashboard for monitoring multiple cameras, with option to call security Ability to download archive footage to MP4 Offers an OpenAI powered assistant that provides contextual support. The bot is fed real-time information about the ongoing event and can respond to user queries, such as "What should I do in this situation" if someone has passed out, helping with quick context-aware advice Offers both real-time streaming and uploaded video analysis ### How we built it Our tech stack combines modern tools for a robust, scalable solution: Frontend: The UI is built with Next.js 13+ and TypeScript, paired with Tailwind CSS for a sleek, responsive design. This ensures a seamless experience for users across different devices. Backend: We use Supabase for secure user authentication and database management, allowing for easy access control and efficient data handling. AI Processing: HawkWatch uses Google's Gemini Visual Language Model (VLM) for real-time video analysis and TensorFlow.js for processing video streams on the client side. These models enable accurate event detection, ranging from criminal activity to health-related emergencies. Email/Phone Service: Resend API powers our email and phone notification system, ensuring that alerts are sent in real-time with minimal delays. Real-time Updates: We leverage the Canvas API for live updates, ensuring that HawkWatch’s real-time analysis is fast and accurate, even as it processes multiple video streams. Contextual Assistance: OpenAI’s language models are integrated to power our assistant bot, which helps security teams with situational guidance. The bot uses context from the most recent events to offer real-time advice, improving the decision-making process during critical moments. ### Challenges we ran into Performance Optimization: Balancing real-time video processing with browser performance and Gemini rate limits AI Model Accuracy: Fine-tuning detection algorithms to minimize false positives Video Stream Handling: Managing multiple video streams without overwhelming the system ### Accomplishments we're proud of Created a fully functional AI surveillance system in 36 hours Achieved real-time processing with minimal latency Implemented a beautiful, intuitive user interface Built a scalable architecture that can handle multiple cameras Developed a system that's accessible through any modern browser ### What we learned Advanced video processing techniques in the browser Real-time data handling with WebSocket connections to handle real-time updates effectively AI model optimization for edge cases Complex state management in React applications, especially when dealing with large datasets Integration of multiple third-party services The importance of user experience in security applications ### What's next Future enhancements we're planning: 1. Advanced AI Features Person identification and recognition Object tracking across multiple cameras Behavioral pattern analysis 2. Enhanced Security End-to-end encryption GDPR compliance tools Advanced access control 3. Smart Home Integration Integration with popular smart home platforms Automated response actions Voice assistant compatibility Our vision is to make HawkWatch the go-to platform for intelligent video surveillance, making security monitoring more efficient and effective for everyone. ## README (from the GitHub repository) # HawkWatch - AI-Powered Security Surveillance ![Gif 1](public/gifs/landing.gif) ![Gif 2](public/gifs/gallary.gif) ## Inspiration In an era where security cameras are everywhere but meaningful surveillance is scarce, we saw an opportunity to transform passive recording systems into intelligent security guardians. Our inspiration came from real-world incidents where crucial moments were missed despite having camera coverage, and the overwhelming challenge security personnel face in monitoring multiple video feeds simultaneously. We wanted to create a solution that doesn't just record but understands, analyzes, and acts, whether it's for local businesses like grocery markets to bigger organizations like hospitals and shopping malls. ## What it does HawkWatch is an intelligent video surveillance platform that detects crime, suspicious activities and life threatening events such as fainting and choking and sends phone alerts to alert security of the issue. Our intelligent model generates time-stamped incident reports with video evidence. It has 3 main features: 1. Real-time analysis of video streams using Google's Gemini Visual Language Model 2. An upload feature that uploads an existing mp4 file for crime analysis 3. A library of saved livestream footage and mp4 uploads, with detailed security analysis complete with timeline and information which is saved with each entry ### Additional features * Sends instant alerts to security through email/phone notifications * Provides an intuitive dashboard for monitoring multiple cameras * Offers an OpenAI powered assistant that provides contextual support. The bot is fed real-time information about the ongoing event and can respond to user queries, such as "What should I do in this situation" if someone has passed out, helping with quick context-aware advice * Offers both real-time streaming and uploaded video analysis * Statistics page which offers an AI summary, chart analysis, and the option to export to CSV. ## How we built it Our tech stack combines modern tools for a robust, scalable solution: * **Frontend**: The UI is built with Next.js 13+ and TypeScript, paired with Tailwind CSS for a sleek, responsive design. This ensures a seamless experience for users across different devices. * **Backend**: We use Supabase for secure user authentication and database management, allowing for easy access control and efficient data handling. * **AI Processing**: HawkWatch uses Google's Gemini Visual Language Model (VLM) for real-time video analysis and TensorFlow.js for processing video streams on the client side. These models enable accurate event detection, ranging from criminal activity to health-related emergencies. * **Email/Phone Service**: Resend API powers our email and phone notification system, ensuring that alerts are sent in real-time with minimal delays. * **Real-time Updates**: We leverage the Canvas API for live updates, ensuring that HawkWatch’s real-time analysis is fast and accurate, even as it processes multiple video streams. * **Contextual Assistance**: OpenAI’s language models are integrated to power our assistant bot, which helps security teams with situational guidance. The bot uses context from the most recent events to offer real-time advice, improving the decision-making process during critical moments. ## Challenges we ran into 1. **Performance Optimization**: Balancing real-time video processing with browser performance and Gemini rate limits 2. **AI Model Accuracy**: Fine-tuning detection algorithms to minimize false positives 3. **Video Stream Handling**: Managing multiple video streams without overwhelming the system ## Accomplishments that we're proud of * Created a fully functional AI surveillance system in 36 hours * Achieved real-time processing with minimal latency * Implemented a beautiful, intuitive user interface * Built a scalable architecture that can handle multiple cameras * Developed a system that's accessible through any modern browser ## What we learned * Advanced video processing techniques in the browser * Real-time data handling with WebSocket connections to handle real-time updates effectively * AI model optimization for edge cases * Complex state management in React applications, especially when dealing with large datasets * Integration of multiple third-party services * The importance of user experience in security applications ## What's next for HawkWatch Future enhancements we're planning: ### 1. Advanced AI Features * Person identification and recognition * Object tracking across multiple cameras * Behavioral pattern analysis ### 2. Enhanced Security * End-to-end encryption * GDPR compliance tools * Advanced access control ### 3. Smart Home Integration * Integration with popular smart home platforms * Automated response actions * Voice assistant compatibility Our vision is to make HawkWatch the go-to platform for intelligent video surveillance, making security monitoring more efficient and effective for everyone. ## Detected evidence (automated analysis) Indexed codebase: 91 recognized source files, 234 KB. - CSS (language) — detected in the code - Google Gemini (technology) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Supabase (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code - Vercel (technology) — detected in the code - TensorFlow (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (110 of 110) ``` .gitignore app/(auth-pages)/forgot-password/page.tsx app/(auth-pages)/layout.tsx app/(auth-pages)/sign-in/page.tsx app/(auth-pages)/sign-up/page.tsx app/(auth-pages)/smtp-message.tsx app/actions.ts app/api/analyze/route.ts app/api/chat/route.ts app/api/send-email/route.ts app/api/summary/route.ts app/api/upload/route.ts app/auth/callback/route.ts app/components/Timeline.tsx app/globals.css app/layout.tsx app/page.tsx app/pages/realtimeStreamPage/actions.ts app/pages/realtimeStreamPage/page.tsx app/pages/saved-videos/page.tsx app/pages/statistics/page.tsx app/pages/upload/actions.ts app/pages/upload/page.tsx app/pages/uploadOneVideoPage/page.tsx app/pages/video/[id]/page.tsx app/protected/page.tsx app/protected/reset-password/page.tsx app/test/testEmail/page.tsx app/test/testJsonBoxesPage/page.tsx app/test/testJsonBoxesVideo/page.tsx app/types.ts app/types/speech-recognition.d.ts backup/gemini.ts backup/page.tsx components.json components/animated-text.tsx components/bounding-boxes-overlay.tsx components/boundingBoxDrawer.tsx components/camera-feed.tsx components/camera-modal.tsx components/chat-interface.tsx components/date-picker.tsx components/deploy-button.tsx components/env-var-warning.tsx components/event-feed.tsx components/form-message.tsx components/gemini-footer.tsx components/header-auth.tsx components/header-nav.tsx components/hero.tsx components/home-link.tsx components/main-navbar.tsx components/navigation-events.tsx components/next-logo.tsx components/particle-background.tsx components/security-alert-modal.tsx components/stats-overview.tsx components/submit-button.tsx components/supabase-logo.tsx components/theme-switcher.tsx components/timestamp-list.tsx components/tutorial/code-block.tsx components/tutorial/connect-supabase-steps.tsx components/tutorial/fetch-data-steps.tsx components/tutorial/sign-up-user-steps.tsx components/tutorial/tutorial-step.tsx components/typography/inline-code.tsx components/ui/badge.tsx components/ui/button.tsx components/ui/checkbox.tsx components/ui/dialog.tsx components/ui/dropdown-menu.tsx components/ui/input.tsx components/ui/label.tsx components/ui/progress.tsx components/ui/uploadFileButton.tsx components/ui/visually-hidden.tsx components/video-player.tsx lib/data.ts lib/utils.ts lib/video_analysis_raw.txt middleware.ts next.config.ts package.json postcss.config.js public/bounding_boxes/Fighting0_boxes.json public/bounding_boxes/Fighting1_boxes.json public/bounding_boxes/Fighting2_boxes.json public/bounding_boxes/Fighting3_boxes.json public/bounding_boxes/Robbery1_boxes.json public/bounding_boxes/Robbery2_boxes.json public/bounding_boxes/Robbery3_boxes.json public/bounding_boxes/Shoplifting0_boxes.json public/bounding_boxes/Shoplifting1_boxes.json public/bounding_boxes/Shoplifting2_boxes.json public/bounding_boxes/Stealing1_boxes.json public/bounding_boxes/Vandalism3_boxes.json README.md scripts/generate_bounding_boxes.py scripts/generate_video_data.py scripts/requirements_detection.txt scripts/requirements.txt tailwind.config.ts tsconfig.json types/index.ts utils/supabase/check-env-vars.ts utils/supabase/client.ts utils/supabase/middleware.ts utils/supabase/server.ts utils/utils.ts ``` ### Dependencies - package.json: @google-cloud/video-intelligence@^5.3.0, @google/generative-ai@^0.21.0, @radix-ui/react-checkbox@^1.1.1, @radix-ui/react-dialog@^1.1.6, @radix-ui/react-dropdown-menu@^2.1.1, @radix-ui/react-label@^2.1.0, @radix-ui/react-progress@^1.1.2, @radix-ui/react-slot@^1.1.0, @supabase/ssr@^0.5.2, @supabase/supabase-js@latest, @tanstack/react-table@^8.21.2, @tensorflow-models/blazeface@^0.1.0, @tensorflow-models/pose-detection@^2.1.3, @tensorflow/tfjs@^4.22.0, @types/classnames@^2.3.4, @types/node@22.10.2, @types/nprogress@^0.2.3, @types/react@^19.0.2, @types/react-dom@19.0.2, @vercel/blob@^0.27.1, autoprefixer@10.4.20, chart.js@^4.4.7, class-variance-authority@^0.7.0, classnames@^2.5.1, clsx@^2.1.1, date-fns@^4.1.0, framer-motion@^12.4.3, lucide-react@^0.468.0, next@latest, next-themes@^0.4.3, nprogress@^0.2.0, openai@^4.85.1, postcss@8.4.49, prettier@^3.3.3, radix-ui@^1.1.3, react@19.0.0, react-chartjs-2@^5.3.0, react-dom@19.0.0, react-particles@^2.12.2, resend@^4.1.2, supabase@^2.12.1, tailwind-merge@^2.6.0, tailwindcss@3.4.17, tailwindcss-animate@^1.0.7, tsparticles-engine@^2.12.0, tsparticles-slim@^2.12.0, typescript@5.7.2, vercel@^41.1.4 - scripts/requirements.txt: google-generativeai@>=0.3.0 ### Recent commits (newest first) - Updated Readme - Fixed upload video feature - Increased prompting for gemini - Fixed realtime mp4 saving conversion issue as well as updated upload gemini prompt - Edited readme - Made loading bar fast - fixed loading bar - Edited landing page phrases - added loading bar - Added AI analysis summary - added timeline to individual page - added timeline - Merge branch 'main' of github.com:Grace-Shao/Treehacks2025 - Added mp4 download feature - made hawkwatch the title - Added mp4 download feature - added dangerous vs non dangerous moments, gemini footer - Merge branch 'main' of github.com:Grace-Shao/Treehacks2025 - Added openai chatbot - added CSV download button ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start" }, "dependencies": { "@google-cloud/video-intelligence": "^5.3.0", "@google/generative-ai": "^0.21.0", "@radix-ui/react-checkbox": "^1.1.1", "@radix-ui/react-dialog": "^1.1.6", "@radix-ui/react-dropdown-menu": "^2.1.1", "@radix-ui/react-label": "^2.1.0", "@radix-ui/react-progress": "^1.1.2", "@radix-ui/react-slot": "^1.1.0", "@supabase/ssr": "^0.5.2", "@supabase/supabase-js": "latest", "@tanstack/react-table": "^8.21.2", "@tensorflow-models/blazeface": "^0.1.0", "@tensorflow-models/pose-detection": "^2.1.3", "@tensorflow/tfjs": "^4.22.0", "@types/classnames": "^2.3.4", "@types/nprogress": "^0.2.3", "@vercel/blob": "^0.27.1", "autoprefixer": "10.4.20", "chart.js": "^4.4.7", "class-variance-authority": "^0.7.0", "classnames": "^2.5.1", "clsx": "^2.1.1", "date-fns": "^4.1.0", "framer-motion": "^12.4.3", "lucide-react": "^0.468.0", "next": "latest", "next-themes": "^0.4.3", "nprogress": "^0.2.0", "openai": "^4.85.1", "prettier": "^3.3.3", "radix-ui": "^1.1.3", "react": "19.0.0", "react-chartjs-2": "^5.3.0", "react-dom": "19.0.0", "react-particles": "^2.12.2", "resend": "^4.1.2", "supabase": "^2.12.1", "tsparticles-engine": "^2.12.0", "tsparticles-slim": "^2.12.0", "vercel": "^41.1.4" }, "devDependencies": { "@types/node": "22.10.2", "@types/react": "^19.0.2", "@types/react-dom": "19.0.2", "postcss": "8.4.49", "tailwind-merge": "^2.6.0", "tailwindcss": "3.4.17", "tailwindcss-animate": "^1.0.7", "typescript": "5.7.2" } } ``` ### scripts/requirements.txt ``` google-generativeai>=0.3.0 ``` ### types/index.ts ```typescript export interface Camera { id: string name: string location: string address: string thumbnail: string videoUrl?: string } export interface Location { id: string name: string cameras: Camera[] } export interface Event { id: string camera: Camera type: string timestamp: Date thumbnail?: string description?: string } export interface BoundingBoxData { video_info: { name: string width: number height: number fps: number total_frames: number frame_interval: number } frames: { [frameNumber: string]: { boxes: [number, number, number, number][] confidences: number[] is_keyframe: boolean } } } ``` ### app/page.tsx ```typescript import Hero from "@/components/hero"; import ConnectSupabaseSteps from "@/components/tutorial/connect-supabase-steps"; import SignUpUserSteps from "@/components/tutorial/sign-up-user-steps"; import { hasEnvVars } from "@/utils/supabase/check-env-vars"; import Link from "next/link" import ParticleBackground from "@/components/particle-background" import AnimatedText from "@/components/animated-text" export default async function Home() { return (

HawkWatch

Get Started
) } ``` ### app/layout.tsx ```typescript import DeployButton from "@/components/deploy-button"; import { EnvVarWarning } from "@/components/env-var-warning"; import HeaderAuth from "@/components/header-auth"; import { hasEnvVars } from "@/utils/supabase/check-env-vars"; import { Geist } from "next/font/google"; import { ThemeProvider } from "next-themes"; import Link from "next/link"; import HomeLink from "@/components/home-link"; import { HeaderNav } from "@/components/header-nav"; import { GeminiFooter } from "@/components/gemini-footer"; import "./globals.css"; import "nprogress/nprogress.css"; import { NavigationEvents } from "@/components/navigation-events"; import NProgress from "nprogress"; // Configure NProgress to complete instantly NProgress.configure({ showSpinner: false, trickleSpeed: 1, minimum: 0.99, easing: 'ease', speed: 1 }); const defaultUrl = process.env.VERCEL_URL ? `https://${process.env.VERCEL_URL}` : "http://localhost:3000"; export const metadata = { metadataBase: new URL(defaultUrl), title: "Hawkwatch", description: "Real-time workplace safety monitoring and analysis", }; const geistSans = Geist({ display: "swap", subsets: ["latin"], }); export default function RootLayout({ children, }: Readonly<{ children: React.ReactNode; }>) { return (
{children}
); } ``` ### app/(auth-pages)/layout.tsx ```typescript export default async function Layout({ children, }: { children: React.ReactNode; }) { return (
{children}
); } ``` ### utils/supabase/server.ts ```typescript import { createServerClient } from "@supabase/ssr"; import { cookies } from "next/headers"; export const createClient = async () => { const cookieStore = await cookies(); return createServerClient( process.env.NEXT_PUBLIC_SUPABASE_URL!, process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY!, { cookies: { getAll() { return cookieStore.getAll(); }, setAll(cookiesToSet) { try { cookiesToSet.forEach(({ name, value, options }) => { cookieStore.set(name, value, options); }); } catch (error) { // The `set` method was called from a Server Component. // This can be ignored if you have middleware refreshing // user sessions. } }, }, }, ); }; ``` ### app/protected/page.tsx ```typescript "use client" import { useState } from "react" import { createClient } from "@/utils/supabase/client" import { redirect } from "next/navigation" import { CameraFeed } from "@/components/camera-feed" import { CameraModal } from "@/components/camera-modal" import { EventFeed } from "@/components/event-feed" import { StatsOverview } from "@/components/stats-overview" import { locations, events } from "@/lib/data" export default function ProtectedPage() { const supabase = createClient() const [selectedCamera, setSelectedCamera] = useState(null) const [videoTimes, setVideoTimes] = useState>({}) const [hoveredCamera, setHoveredCamera] = useState(null) const handleAuth = async () => { const { data: { user }, } = await supabase.auth.getUser() if (!user) { return redirect("/sign-in") } } // Check auth on mount useState(() => { handleAuth() }) const handleTimeUpdate = (cameraId: string, time: number) => { setVideoTimes(prev => ({ ...prev, [cameraId]: time })) } const handleEventClick = (cameraId: string, timestamp: number) => { setSelectedCamera(cameraId) // Update the video time for this camera to jump to the incident setVideoTimes(prev => ({ ...prev, [cameraId]: timestamp })) } return (
{/* Main Content */}
{locations.flatMap((location) => location.cameras.map((camera) => ( )) )}
{/* Sidebar */}
{/* Camera Modal */} {selectedCamera && ( !open && setSelectedCamera(null)} cameraId={selectedCamera} currentTime={videoTimes[selectedCamera]} date={new Date()} /> )}
) } ``` ### app/api/upload/route.ts ```typescript import { put } from "@vercel/blob" import { NextResponse } from "next/server" export async function POST(request: Request) { try { const formData = await request.formData() const file = formData.get("file") as File if (!file) { return NextResponse.json({ error: "No file provided" }, { status: 400 }) } const blob = await put(file.name, file, { access: "public", }) return NextResponse.json({ url: blob.url }) } catch (error) { return NextResponse.json({ error: "Error uploading file" }, { status: 500 }) } } ``` ### app/auth/callback/route.ts ```typescript import { createClient } from "@/utils/supabase/server"; import { NextResponse } from "next/server"; export async function GET(request: Request) { // The `/auth/callback` route is required for the server-side auth flow implemented // by the SSR package. It exchanges an auth code for the user's session. // https://supabase.com/docs/guides/auth/server-side/nextjs const requestUrl = new URL(request.url); const code = requestUrl.searchParams.get("code"); const origin = requestUrl.origin; const redirectTo = requestUrl.searchParams.get("redirect_to")?.toString(); if (code) { const supabase = await createClient(); await supabase.auth.exchangeCodeForSession(code); } if (redirectTo) { return NextResponse.redirect(`${origin}${redirectTo}`); } // URL to redirect to after sign up process completes return NextResponse.redirect(`${origin}/pages/mainOptions`); } ``` [80 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]