# Project export: Blaize - Realtime Intelligent Wildfire Information This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Blaize is a mobile-first web-app that provides users with intelligent, responsive updates on active wildfires. We feature a smart AI assistant with bidirectional real-time voice communications. - Devpost: https://devpost.com/software/blaize-realtime-intelligent-wildfire-information - GitHub: https://github.com/gdodge0/WildfireAgent - Team: 3 GitHub contributor(s) — Gabriel Dodge (53 commits), sradisay (49 commits), Evan (27 commits) ## Devpost submission (written by the team) ### Inspiration Last month, the three of us had our camping trip in the San Bernardino National Forest cut short by the Line Fire as it exploded in size overnight. As we awaited evacuation notices and warnings from official sources, we found ourselves frequently checking different sources for the latest information, which became a hassle as we raced against the clock to pack up our camping gear and prepared to evacuate. ### What it does Blaize automatically gathers and compiles information from authoritative sources, and stores the latest information in one place. Not only can users can view the latest notices and evacuation information from a single app, but they can text or speak with our intelligent AI agent which always has access to a rich set of information in addition to the latest updates from authorities. ### How we built it We built Blaize using a Flask backend for retrieving and compiling information from the WatchDuty API, and utilizes redis for quick information caching. The frontend was built in Vue.js with Tailwind CSS, and the AI agent is provided through Deepgram, based on Google's Gemini-1.5. The AI agent is fed the latest information in a carefully crafted prompt through our Flask API, and can answer user's questions pertaining to a specific wildfire. ### Challenges we ran into We faced slow response times, mostly due to poor wifi connection during development. This made us realize, however, that this simulated an environment closer to the app's real use case, where users may not have access to good cell service. We then implemented a Redis cache to improve load times. ### Accomplishments we're proud of Speaking to or texting the AI agent feels responsive, informative, and realistic, akin to speaking with a trusted expert. We are proud to have built an agent that will quickly and concisely respond to those in affected wildfire areas, which will help to provide information during periods where emergency services are often constrained. ### What we learned We learned the importance of considering real-world conditions during development, such as limited connectivity, which prompted us to implement caching strategies to enhance performance. We also gained valuable experience integrating various technologies, such as combining real-time data APIs with an AI agent to provide dynamic and contextually relevant information. ### What's next We would like to iterate on and streamline our backend, and develop a feature-complete mobile app for iOS and Android. After achieving this, we hope to get in contact with wildfire information sources such as WatchDuty for new integrations and potential collaborations. ## README (from the GitHub repository) No README available. ## Detected evidence (automated analysis) Indexed codebase: 42 recognized source files, 67 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - Redis (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - Vue (technology) — detected in the code - Google Gemini (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (53 of 53) ``` .github/workflows/node.js.yml .gitignore api/watchduty.py app.py Dockerfile-flask Dockerfile-websockets helpers.py migrations/alembic.ini migrations/env.py migrations/README migrations/script.py.mako migrations/versions/c84c0ed373b9_initial_migration.py models/models.py prompt.txt requirements.txt websocket_server.py wildfireagent_frontend/.gitignore wildfireagent_frontend/index.html wildfireagent_frontend/jsconfig.json wildfireagent_frontend/package.json wildfireagent_frontend/postcss.config.js wildfireagent_frontend/README.md wildfireagent_frontend/src/App.vue wildfireagent_frontend/src/assets/base.css wildfireagent_frontend/src/assets/main.css wildfireagent_frontend/src/assets/tailwind.css wildfireagent_frontend/src/components/Chat.vue wildfireagent_frontend/src/components/ChatAudio.vue wildfireagent_frontend/src/components/Dashboard.vue wildfireagent_frontend/src/components/Detail.vue wildfireagent_frontend/src/components/FireItem.vue wildfireagent_frontend/src/components/GoogleMap.vue wildfireagent_frontend/src/components/HelloWorld.vue wildfireagent_frontend/src/components/icons/IconCommunity.vue wildfireagent_frontend/src/components/icons/IconDocumentation.vue wildfireagent_frontend/src/components/icons/IconEcosystem.vue wildfireagent_frontend/src/components/icons/IconFlame.vue wildfireagent_frontend/src/components/icons/IconSupport.vue wildfireagent_frontend/src/components/icons/IconTooling.vue wildfireagent_frontend/src/components/TheWelcome.vue wildfireagent_frontend/src/components/WatchingItem.vue wildfireagent_frontend/src/components/WelcomeItem.vue wildfireagent_frontend/src/DashboardPage.vue wildfireagent_frontend/src/DetailPage.vue wildfireagent_frontend/src/main.js wildfireagent_frontend/src/router/index.js wildfireagent_frontend/src/utils/api.js wildfireagent_frontend/src/utils/browser.js wildfireagent_frontend/src/utils/chatSession.js wildfireagent_frontend/src/utils/location.js wildfireagent_frontend/src/utils/search.js wildfireagent_frontend/tailwind.config.js wildfireagent_frontend/vite.config.js ``` ### Dependencies - requirements.txt: aenum@==3.1.15, aiofiles@==24.1.0, aiohappyeyeballs@==2.4.3, aiohttp@==3.10.11, aiosignal@==1.3.1, alembic@==1.13.3, annotated-types@==0.7.0, anyio@==4.6.2.post1, attrs@==24.2.0, beautifulsoup4@==4.12.3, blinker@==1.8.2, cachetools@==5.5.0, certifi@==2024.8.30, charset-normalizer@==3.4.0, click@==8.1.7, dataclasses-json@==0.6.7, deepgram-sdk@==3.7.4, deprecation@==2.1.0, distro@==1.9.0, Flask@==3.0.3, Flask-Cors@==5.0.0, Flask-Migrate@==4.0.7, Flask-SQLAlchemy@==3.1.1, frozenlist@==1.4.1, google-ai-generativelanguage@==0.6.10, google-api-core@==2.21.0, google-api-python-client@==2.149.0, google-auth@==2.35.0, google-auth-httplib2@==0.2.0, google-generativeai@==0.8.3, googleapis-common-protos@==1.65.0, greenlet@==3.1.1, grpcio@==1.67.0, grpcio-status@==1.67.0, h11@==0.14.0, httpcore@==1.0.6, httplib2@==0.22.0, httpx@==0.27.2, idna@==3.10, itsdangerous@==2.2.0, janus@==1.0.0, Jinja2@==3.1.4, jiter@==0.6.1, Mako@==1.3.6, MarkupSafe@==3.0.2, marshmallow@==3.23.0, multidict@==6.1.0, mypy-extensions@==1.0.0, openai@==1.52.0, packaging@==24.1, propcache@==0.2.0, proto-plus@==1.24.0, protobuf@==5.28.2, psycopg2-binary@==2.9.10, pyasn1@==0.6.1, pyasn1_modules@==0.4.1, PyAudio@==0.2.14, pydantic@==2.9.2, pydantic_core@==2.23.4, pyparsing@==3.2.0, python-dotenv@==1.0.1, python-redis-cache@==4.0.1, pytz@==2024.2, redis@==5.2.0, requests@==2.32.3, rsa@==4.9, sniffio@==1.3.1, soupsieve@==2.6, SQLAlchemy@==2.0.36, tqdm@==4.66.5, typing_extensions@==4.12.2, typing-inspect@==0.9.0, uritemplate@==4.1.1, urllib3@==2.2.3, waitress@==3.0.1, websockets@==13.1, Werkzeug@==3.0.6, yarl@==1.15.5 - wildfireagent_frontend/package.json: @vitejs/plugin-vue@^5.1.4, autoprefixer@^10.4.20, axios@^1.7.7, marked@^14.1.3, postcss@^8.4.47, tailwindcss@^3.4.14, vite@^5.4.8, vue@^3.5.12, vue-router@^4.4.5, vue3-google-map@^0.21.0 ### Recent commits (newest first) - bugfix - quick test - bugfix - add localized timezones - fix time format - Update requirements.txt - Merge pull request #12 from gdodge0/dependabot/pip/aiohttp-3.10.11 - Bump aiohttp from 3.10.10 to 3.10.11 - Update node.js.yml - Merge pull request #10 from gdodge0/dependabot/pip/werkzeug-3.0.6 - Merge pull request #11 from gdodge0/dependabot/pip/waitress-3.0.1 - Bump waitress from 3.0.0 to 3.0.1 - Bump werkzeug from 3.0.4 to 3.0.6 - add env vars for WebSocket and API urls - add env vars for WebSocket and API urls - add prod env - add prod env - fix api key - fix api key - fix env dir ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` aenum==3.1.15 aiofiles==24.1.0 aiohappyeyeballs==2.4.3 aiohttp==3.10.11 aiosignal==1.3.1 annotated-types==0.7.0 anyio==4.6.2.post1 attrs==24.2.0 beautifulsoup4==4.12.3 blinker==1.8.2 cachetools==5.5.0 certifi==2024.8.30 charset-normalizer==3.4.0 click==8.1.7 dataclasses-json==0.6.7 deepgram-sdk==3.7.4 deprecation==2.1.0 distro==1.9.0 Flask==3.0.3 Flask-SQLAlchemy==3.1.1 frozenlist==1.4.1 google-ai-generativelanguage==0.6.10 google-api-core==2.21.0 google-api-python-client==2.149.0 google-auth==2.35.0 google-auth-httplib2==0.2.0 google-generativeai==0.8.3 googleapis-common-protos==1.65.0 grpcio==1.67.0 grpcio-status==1.67.0 h11==0.14.0 httpcore==1.0.6 httplib2==0.22.0 httpx==0.27.2 idna==3.10 itsdangerous==2.2.0 janus==1.0.0 Jinja2==3.1.4 jiter==0.6.1 MarkupSafe==3.0.2 marshmallow==3.23.0 multidict==6.1.0 mypy-extensions==1.0.0 openai==1.52.0 packaging==24.1 propcache==0.2.0 proto-plus==1.24.0 protobuf==5.28.2 pyasn1==0.6.1 pyasn1_modules==0.4.1 PyAudio==0.2.14 pydantic==2.9.2 pydantic_core==2.23.4 pyparsing==3.2.0 python-dotenv==1.0.1 requests==2.32.3 rsa==4.9 sniffio==1.3.1 soupsieve==2.6 SQLAlchemy==2.0.36 tqdm==4.66.5 typing-inspect==0.9.0 typing_extensions==4.12.2 uritemplate==4.1.1 urllib3==2.2.3 websockets==13.1 Werkzeug==3.0.6 yarl==1.15.5 ## The following requirements were added by pip freeze: alembic==1.13.3 Flask-Cors==5.0.0 Flask-Migrate==4.0.7 greenlet==3.1.1 Mako==1.3.6 psycopg2-binary==2.9.10 python-redis-cache==4.0.1 pytz==2024.2 redis==5.2.0 waitress==3.0.1 ``` ### wildfireagent_frontend/package.json ``` { "name": "wildfireagent", "version": "0.0.0", "private": true, "type": "module", "scripts": { "dev": "vite", "build": "vite build", "preview": "vite preview" }, "dependencies": { "axios": "^1.7.7", "marked": "^14.1.3", "vue": "^3.5.12", "vue-router": "^4.4.5", "vue3-google-map": "^0.21.0" }, "devDependencies": { "@vitejs/plugin-vue": "^5.1.4", "autoprefixer": "^10.4.20", "postcss": "^8.4.47", "tailwindcss": "^3.4.14", "vite": "^5.4.8" } } ``` ### app.py ```python import json import logging import os from flask_cors import CORS, cross_origin from helpers import * from flask import Flask, send_from_directory, jsonify, request from models.models import ChatSession, db from dotenv import load_dotenv from api.watchduty import get_fire_summary, get_current_fires from flask_migrate import Migrate load_dotenv() class Config: SQLALCHEMY_TRACK_MODIFICATIONS = False SQLALCHEMY_ENGINE_OPTIONS = { "pool_pre_ping": True, "pool_recycle": 600 } SQLALCHEMY_DATABASE_URI = os.environ.get("SQLALCHEMY_DATABASE_URI") app = Flask(__name__, static_folder="./dist", static_url_path="/dist") app.config.from_object(Config) migrate = Migrate(app, db) db.init_app(app) CORS(app) @app.route('/') @app.route('/detail/') def index(id=None): return send_from_directory('dist', 'index.html') @cross_origin() @app.route('/api/v1/start_chat_session') def start_LLM_session(): geo_id = request.args["geo_id"] app.logger.debug("creating session") summary = get_fire_summary(geo_id) messages = [ { "msg_id": 0, "type": "system_message", "description": "important context for this chat session", "data": summary } ] chat_session = ChatSession(json.dumps(messages)) db.session.add(chat_session) db.session.commit() return jsonify({ "success": True, "session_id": chat_session.id, "event_data": summary["messages"]["geo_events"][0] }) @app.route('/api/v1/get_msg_history') def msg_history(): session_id = request.args["session_id"] session = ChatSession.query.filter_by(id=session_id).one() messages = [] for message in json.loads(session.chat_ctx): if message["msg_id"] == 0: continue messages.append(message) return jsonify(messages) @app.route('/api/v1/get_fire_info') def get_fire_info(): return jsonify(get_current_fires()) @app.route('/api/v1/get_single_info') @cross_origin() def get_single_info(): return jsonify(get_fire_summary(request.args["geo_id"])) @app.route('/api/v1/get_latest_news_batch', methods=['GET']) def get_latest_news_batch(): # Get 'fire_ids' from the query string tz = request.args["tz"] fire_ids = request.args.getlist('fire_ids') if not fire_ids: return jsonify({"error": "fire_ids parameter is required"}), 400 news_data = [] for fire_id in fire_ids: summary = get_fire_summary(fire_id) news_data.append({ "id": fire_id, "headline": summary["messages"]["summary"][0]["message"], "time": pretty_date_time(summary["messages"]["summary"][0]["timestamp"], tz) }) return jsonify(news_data), 200 @app.route("/") def serve_others(filename): return send_from_directory(app.static_folder, filename) @app.route("/assets/") def serve_image(filename): return send_from_directory(app.static_folder, "assets/" + filename) @app.route("/sample", methods=["GET"]) def serve_index(): return app.send_static_file("index.html") @app.route("/overflow", methods=["GET"]) def overflow(): return app.send_static_file("overflow_test.html") def run_ui(): app.run(debug=True, use_reloader=False) if __name__ == "__main__": logging.basicConfig(level=logging.DEBUG) app.run(debug=True) ``` ### wildfireagent_frontend/src/main.js ```javascript import './assets/tailwind.css' import { createApp } from 'vue' import App from './App.vue' import router from './router' const app = createApp(App); const gmaps_api_key = import.meta.env.VITE_APP_API_KEY const ws_url = import.meta.env.VITE_APP_WS_URL const api_url = import.meta.env.VITE_APP_API_URL app.config.globalProperties.$gmaps_api_key = gmaps_api_key app.config.globalProperties.$ws_url = ws_url app.config.globalProperties.$api_url = api_url app.use(router) app.mount('#app') ``` ### wildfireagent_frontend/src/router/index.js ```javascript import { createRouter, createWebHistory } from 'vue-router'; import DetailPage from "@/DetailPage.vue"; import DashboardPage from "@/DashboardPage.vue"; const routes = [ { path: '/', name: "Dashboard", component: DashboardPage }, { path: '/detail/:id', name: "Detail", component: DetailPage, props: true } ]; const router = createRouter({ history: createWebHistory(), routes, }); export default router; ``` ### helpers.py ```python from datetime import datetime import pytz def pretty_date_time(date_str, tz='America/Los_Angeles'): # Parse the string as UTC time utc = pytz.utc dt = datetime.strptime(date_str, "%Y-%m-%dT%H:%M:%SZ") dt_utc = utc.localize(dt) # Localize as UTC time # Convert UTC to PST (Pacific Standard Time) local_tz = pytz.timezone(tz) dt_localized = dt_utc.astimezone(local_tz) # Get current date in PST now_pst = datetime.now(local_tz) # Determine if the date is today if dt_localized.date() == now_pst.date(): day_str = "today" else: day_str = dt_localized.strftime("%m/%d") # Format the time in a 12-hour format with AM/PM time_str = dt_localized.strftime("%I:%M %p").lstrip('0') # remove leading zero # Combine time and day information if day_str == "today": return f"{time_str} {day_str}" else: return f"{time_str} on {day_str}" ``` ### websocket_server.py ```python import base64 import json import time import httpx from datetime import datetime from websockets.sync.server import serve from models.models import ChatSession, db import os import google.generativeai as genai from dotenv import load_dotenv from app import app from deepgram import ( DeepgramClient, SpeakWSOptions, SpeakWebSocketEvents, FileSource, PrerecordedOptions ) load_dotenv() SYSTEM_INSTRUCTIONS = open('prompt.txt', 'r').read() genai.configure(api_key=os.environ['GOOGLE_API_KEY']) model = genai.GenerativeModel("gemini-1.5-pro", system_instruction=SYSTEM_INSTRUCTIONS) def hello(websocket): # Connect to Deepgram connected = False deepgram = DeepgramClient() dg_connection = deepgram.speak.websocket.v("1") global last_time last_time = time.time() - 5 def on_open(self, open, **kwargs): print(f"\n\n{open}\n\n") def on_flush(self, flushed, **kwargs): print(f"\n\n{flushed}\n\n") flushed_str = str(flushed) websocket.send(flushed_str) def on_binary_data(self, data, **kwargs): print("Received binary data") global last_time if time.time() - last_time > 3: print("------------ [Binary Data] Attach header.\n") # Add a wav audio container header to the file if you want to play the audio # using the AudioContext or media player like VLC, Media Player, or Apple Music # Without this header in the Chrome browser case, the audio will not play. header = bytes( [ 0x52, 0x49, 0x46, 0x46, # "RIFF" 0x00, 0x00, 0x00, 0x00, # Placeholder for file size 0x57, 0x41, 0x56, 0x45, # "WAVE" 0x66, 0x6D, 0x74, 0x20, # "fmt " 0x10, 0x00, 0x00, 0x00, # Chunk size (16) 0x01, 0x00, # Audio format (1 for PCM) 0x01, 0x00, # Number of channels (1) 0x80, 0xBB, 0x00, 0x00, # Sample rate (48000) 0x00, 0xEE, 0x02, 0x00, # Byte rate (48000 * 2) 0x02, 0x00, # Block align (2) 0x10, 0x00, # Bits per sample (16) 0x64, 0x61, 0x74, 0x61, # "data" 0x00, 0x00, 0x00, 0x00, # Placeholder for data size ] ) websocket.send(header) last_time = time.time() websocket.send(data) def on_close(self, close, **kwargs): print(f"\n\n{close}\n\n") dg_connection.on(SpeakWebSocketEvents.Open, on_open) dg_connection.on(SpeakWebSocketEvents.AudioData, on_binary_data) dg_connection.on(SpeakWebSocketEvents.Flushed, on_flush) dg_connection.on(SpeakWebSocketEvents.Close, on_close) try: while True: message = websocket.recv() # print(f"message from UI: {message}") data = json.loads(message) media = data.get("media") text = data.get("text") session_id = data.get("session_id") voice_model = "aura-asteria-en" with app.app_context(): chat_session = ChatSession.query.filter_by(id=session_id).one() if chat_session is None: continue if media: split_media = media.split(',') media_file = base64.b64decode(split_media[-1]) # Transcription payload: FileSource = { "buffer": media_file, } options: PrerecordedOptions = PrerecordedOptions( model="nova-2", smart_format=True, utterances=True, punctuate=True, diarize=True, ) response = deepgram.listen.rest.v("1").transcribe_file( payload, options, timeout=httpx.Timeout(300.0, connect=10.0) ) utterances = response["results"]["utterances"] if len(utterances) == 0: text = "" else: text = utterances[0]["transcript"] else: if text is None: raise Exception("text and media cannot be blank") # Are we connected to the Deepgram TTS WS? if connected is False: options: SpeakWSOptions = SpeakWSOptions( model=voice_model, encoding="linear16", sample_rate=48000, ) if dg_connection.start(options) is False: if app.debug: app.logger.debug( "Unable to start Deepgram TTS WebSocket connection" ) raise Exception("Unable to start Deepgram TTS WebSocket connection") connected = True current_question = { "msg_id": chat_session.msg_count, "type": "user", "description": "a message from a user.", "data": text } try: llm_prompt = f""" You are in a chat session. [truncated — 2102 more characters] ``` ### wildfireagent_frontend/postcss.config.js ```javascript export default { plugins: { tailwindcss: {}, autoprefixer: {}, }, } ``` ### wildfireagent_frontend/vite.config.js ```javascript import { fileURLToPath, URL } from 'node:url' import { defineConfig } from 'vite' import vue from '@vitejs/plugin-vue' // https://vite.dev/config/ export default defineConfig({ plugins: [ vue(), ], resolve: { alias: { '@': fileURLToPath(new URL('./src', import.meta.url)) } } }) ``` ### wildfireagent_frontend/tailwind.config.js ```javascript /** @type {import('tailwindcss').Config} */ export default { content: [ './index.html', './src/**/*.{vue,js}', './src/**/*.vue', ], theme: { extend: { fontFamily: { sans: ['Inter', 'sans-serif'], // Set Inter as default sans-serif font }, }, }, plugins: [], } ``` [33 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]