# Project export: Campanion This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Campanion is an AI tool that offers personal preventive knowledge and a trusted emergency response system so that you can live in the moment of your trip.. - Devpost: https://devpost.com/software/campanion-pg4ahx - GitHub: https://github.com/ankurgarg-code/calhackscampanion - Team: 2 GitHub contributor(s) — Arnur (7 commits), Ankur Garg (2 commits) ## Devpost submission (written by the team) ### Inspiration As a team of outdoor enthusiasts, we’ve all had those moments where the unexpected happens—getting lost, an injury, or sudden weather changes. We realized how vulnerable we felt without instant access to the right information or emergency help. That shared experience drove us to create Campanion, an AI tool designed to provide real-time guidance and safety tips, so explorers like us can stay prepared and enjoy nature with peace of mind. Our goal is to turn those uncertain moments into empowering ones, giving every adventurer the confidence to explore safely. ### What it does Campanion takes as input the hike you're planning on going on and what items you plan on carrying in your bag during the hike, and then provides personalized safety recommendations, potential hazard alerts, and essential tips based on the terrain, weather conditions, and your equipment. In case of danger, you can trigger an emergency alert to notify your contacts with your location. Plus, you can chat with the bot to learn how to safely navigate the outdoors, getting real-time guidance and advice tailored to your specific situation. ### How we built it Our back-end is powered by Python, integrated with Chroma DB to efficiently manage and retrieve data. For AI capabilities, we utilized Gemini 1.5 alongside Langchain, ensuring fluid and dynamic interactions between users and the AI. The front-end was built using Swift 5 and Swift UI to create a smooth, native iOS experience. To enhance the app’s accuracy and data richness, we incorporated the Weather API and the National Parks Service API. Finally, all design prototypes and user flows were created in Figma to ensure a user-friendly interface. ### Challenges we ran into One of the key challenges we faced while building Campanion was our goal to store the AI model locally on a user’s device. This would allow hikers to still interact with the AI and receive guidance even when they’re in areas without reception, which is critical for outdoor explorers. However, we encountered issues with the Gemini model we were using to power the AI, particularly around optimizing it for offline use while maintaining its responsiveness and accuracy. Balancing the model's size and efficiency without compromising its capabilities remains a technical hurdle we are working through. ### Accomplishments we're proud of One of the accomplishments we're particularly proud of is how we adapted to the challenges we faced, especially with the Gemini model and offline functionality. Initially, we struggled with how to properly scope the project, leading to a lot of back-and-forth adjustments. It was a complex process figuring out the right balance between the model’s performance and ensuring a seamless experience for users even without reception. But through persistence and iteration, we found creative solutions that allowed us to overcome those hurdles and get the app working effectively. We're also proud in how we all rallied to learn skills that we weren't totally familiar with before the hackathon to accomplish that ultimate goal a working version out in the world! ### What we learned One of the key lessons we learned during the development process was how to effectively integrate an LLM into an app and leverage APIs to create a seamless, responsive user experience. This required understanding how to connect the AI model and optimize interactions between the front-end and back-end systems, ensuring that the app functioned smoothly while providing helpful information to users. This technical experience has greatly enhanced our ability to build more integrated and intelligent applications in the future! ### What's next Next for Campanion, we plan to focus on making the app fully functional offline, ensuring users can rely on it even in the most remote areas. We’re also exploring the possibility of integrating satellite technology to provide more accurate, real-time data on weather, terrain, and location, further enhancing the safety and reliability of the app for outdoor adventurers. These developments will make Campanion an even more robust tool for navigating the wilderness. ## README (from the GitHub repository) # Campanion - Your AI-Powered Hiking Assistant ![image](media/Campanion.png) Welcome to **Campanion**, the ultimate AI-driven hiking assistant developed for **Calhacks 11.0**. Our mission is to redefine outdoor safety and enjoyment by offering real-time, intelligent support tailored for hikers. Whether you need help navigating trails, sharing your location in emergencies, or getting critical safety information, Campanion is here to guide and protect you throughout your journey. ## Features - **AI-Powered Chat**: Ask for tips, directions, or safety advice using natural language. Our custom LLM integration ensures you get relevant, context-aware responses. - **Emergency SOS Mode**: Activate emergency mode to notify contacts, share your location, and interact with an AI for immediate assistance. - **Real-Time Alerts**: Receive location-based safety alerts and trail conditions to stay informed about your surroundings. - **Custom Hike Details**: View specific information about the park you're hiking in, including essential safety tips, weather, and emergency contact functionality. - **Packed Item Context**: Use the packed item list to inform the AI assistant about the gear you’re carrying, enabling more personalized guidance. ## Tech Stack ### Frontend - **SwiftUI**: The user interface of Campanion is built with SwiftUI for seamless and elegant designs across iOS devices. - **MapKit**: For location services and providing real-time GPS-based data, we integrate Apple's MapKit. - **CoreLocation**: Used for geolocation services to track hiker movements and provide relevant park information. ### Backend - **Google Gemini & ChromaDB**: We utilize **Google Gemini** as our primary LLM in conjunction with **ChromaDB** for vector storage and retrieval in our RAG (Retrieval-Augmented Generation) pipeline. - **Retrieval-Augmented Generation (RAG)**: To deliver factually accurate and context-rich responses, we implemented RAG, ensuring that users receive answers based on both the LLM’s knowledge and relevant external sources. - **Azure Services**: We leverage Azure cloud services to deploy and manage our backend API, ensuring scalability and reliability. - **Custom LLM Backend**: When the Huggingface Gemma model proved unfeasible for iPhone deployment, we built a custom backend to communicate with our RAG-based LLM, providing context-aware responses and safety tips for hikers. ### API - **Node.js + Express**: Our backend API is built using Node.js with Express, designed to handle requests and communicate with the AI models and databases. - **PostgreSQL**: A PostgreSQL database is used for storing user data, hike details, and park-specific safety information. - **LLM Communication**: The API handles requests from the frontend, querying our AI engine with user questions and contextual information such as the park location and packed items. ## Challenges We Overcame - **Deploying RAG-based LLM**: Initially, we encountered issues with running the Huggingface Gemma model locally on iPhone due to hardware limitations. To solve this, we deployed our own LLM backend, leveraging Google Gemini and ChromaDB to provide accurate and contextually relevant responses. - **Real-Time Emergency Services**: Implementing a robust emergency notification system that seamlessly integrates with satellite connectivity and contacts required careful architecture and testing. - **Dynamic Context Handling**: Passing real-time user context (like location and packed items) to the LLM API was essential for providing hikers with accurate guidance. ## How to Use 1. **Download the App**: Install Campanion on your iPhone and set up your emergency contacts. 2. **Start Your Hike**: Select your hiking location and begin your adventure. Campanion will track your progress and offer real-time guidance. 3. **Stay Safe**: Access essential safety information, real-time alerts, and AI-guided tips based on your hike location and packed items. 4. **SOS Mode**: In case of emergencies, activate SOS mode to notify services and contacts, with satellite support to ensure you’re always connected. ## Conclusion Campanion is more than just a hiking app — it's your AI-powered safety partner on the trail. With a focus on real-time emergency response, personalized guidance, and context-aware interactions, we’ve built a solution that prioritizes hiker safety and enjoyment. Developed for **Calhacks 11.0**, Campanion harnesses the power of cutting-edge AI and real-time technology to redefine outdoor exploration. ## License This project is licensed under the MIT License. --- ## Detected evidence (automated analysis) Indexed codebase: 31 recognized source files, 89 KB. - Flask (technology) — detected in the code - LangChain (technology) — detected in the code - Python (language) — detected in the code - Swift (language) — detected in the code ## Codebase structure (from repository index) ### Files (44 of 44) ``` .DS_Store HikerAI.xcodeproj/project.pbxproj HikerAI.xcodeproj/project.xcworkspace/contents.xcworkspacedata HikerAI.xcodeproj/xcuserdata/arnur.xcuserdatad/xcschemes/xcschememanagement.plist HikerAI/Assets.xcassets/AccentColor.colorset/Contents.json HikerAI/Assets.xcassets/AppIcon.appiconset/Contents.json HikerAI/Assets.xcassets/Contents.json HikerAI/Assets.xcassets/trail_background.imageset/Contents.json HikerAI/HikerAIApp.swift HikerAI/Model/Alerts.swift HikerAI/Model/EmergencyContact.swift HikerAI/Model/HikerDetail.swift HikerAI/Model/HikerDetailItem.swift HikerAI/Model/Park.swift HikerAI/Preview Content/Preview Assets.xcassets/Contents.json HikerAI/Utilities/AlertService.swift HikerAI/Utilities/Extensions.swift HikerAI/Utilities/LocationManager.swift HikerAI/Utilities/ParkService.swift HikerAI/Utilities/Tokenizer.swift HikerAI/View/AddContactsView.swift HikerAI/View/AlertsView.swift HikerAI/View/ChatView.swift HikerAI/View/CityInputScreen.swift HikerAI/View/EmergencyView.swift HikerAI/View/HikeSummaryView.swift HikerAI/View/ItemInputScreen.swift HikerAI/View/LoadingIndicator.swift HikerAI/View/LogoScreen.swift HikerAI/View/MessageRow.swift HikerAI/View/ParkSelection.swift HikerAI/ViewModel/ChatViewModel.swift HikerAI/ViewModel/HikeViewModel.swift HikerAITests/HikerAITests.swift HikerAIUITests/HikerAIUITests.swift HikerAIUITests/HikerAIUITestsLaunchTests.swift rag_model/.DS_Store rag_model/app.py rag_model/campanion_python_standalone.ipynb rag_model/depend/corpus_combined.txt rag_model/depend/README.md rag_model/depend/requirements.txt rag_model/model_setup.py README.md ``` ### Dependencies - rag_model/depend/requirements.txt: Flask@==2.3.2, langchain@==0.0.200, langchain-chroma@==0.0.200, langchain-community@==0.0.200, langchain-google-vertexai@==0.0.200 ### Recent commits (newest first) - Read.me changes - Read.me changes - Commit 1Merge branch 'main' of https://github.com/ankurgarg-code/calhackscampanion - Read.me changes - submission - Submission - Calhacks submission - Initial commit - Initial Commit ## Key source files (fetched from GitHub, selected and truncated for size) ### rag_model/depend/requirements.txt ``` Flask==2.3.2 langchain==0.0.200 # Adjust based on the specific version you are using langchain-chroma==0.0.200 langchain-google-vertexai==0.0.200 langchain-community==0.0.200 ``` ### rag_model/app.py ```python from flask import Flask, request, jsonify from model_setup import get_answer # Import the get_answer function from setup.py # Initialize Flask app app = Flask(__name__) @app.route('/ask', methods=['POST']) def ask_question(): data = request.json question = data.get('question') if not question: return jsonify({"error": "Question is required"}), 400 answer = get_answer(question) return jsonify({"answer": answer}) if __name__ == '__main__': app.run(debug=True) ``` ### HikerAI/HikerAIApp.swift ```swift import SwiftUI @main struct HikerAIApp: App { var body: some Scene { WindowGroup { NavigationStack { LogoScreen() // Start the app at the LogoScreen } } } } ``` ### HikerAITests/HikerAITests.swift ```swift // // HikerAITests.swift // HikerAITests // // Created by Arnur Sabet on 19.10.2024. // import Testing @testable import HikerAI struct HikerAITests { @Test func example() async throws { // Write your test here and use APIs like `#expect(...)` to check expected conditions. } } ``` ### HikerAIUITests/HikerAIUITestsLaunchTests.swift ```swift // // HikerAIUITestsLaunchTests.swift // HikerAIUITests // // Created by Arnur Sabet on 19.10.2024. // import XCTest final class HikerAIUITestsLaunchTests: XCTestCase { override class var runsForEachTargetApplicationUIConfiguration: Bool { true } override func setUpWithError() throws { continueAfterFailure = false } @MainActor func testLaunch() throws { let app = XCUIApplication() app.launch() // Insert steps here to perform after app launch but before taking a screenshot, // such as logging into a test account or navigating somewhere in the app let attachment = XCTAttachment(screenshot: app.screenshot()) attachment.name = "Launch Screen" attachment.lifetime = .keepAlways add(attachment) } } ``` ### HikerAIUITests/HikerAIUITests.swift ```swift // // HikerAIUITests.swift // HikerAIUITests // // Created by Arnur Sabet on 19.10.2024. // import XCTest final class HikerAIUITests: XCTestCase { override func setUpWithError() throws { // Put setup code here. This method is called before the invocation of each test method in the class. // In UI tests it is usually best to stop immediately when a failure occurs. continueAfterFailure = false // In UI tests it’s important to set the initial state - such as interface orientation - required for your tests before they run. The setUp method is a good place to do this. } override func tearDownWithError() throws { // Put teardown code here. This method is called after the invocation of each test method in the class. } @MainActor func testExample() throws { // UI tests must launch the application that they test. let app = XCUIApplication() app.launch() // Use XCTAssert and related functions to verify your tests produce the correct results. } @MainActor func testLaunchPerformance() throws { if #available(macOS 10.15, iOS 13.0, tvOS 13.0, watchOS 7.0, *) { // This measures how long it takes to launch your application. measure(metrics: [XCTApplicationLaunchMetric()]) { XCUIApplication().launch() } } } } ``` ### rag_model/model_setup.py ```python import os from langchain import hub from langchain_chroma import Chroma from langchain_community.document_loaders import TextLoader from langchain_core.output_parsers import StrOutputParser from langchain_core.runnables import RunnablePassthrough from langchain.embeddings import HuggingFaceEmbeddings from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_google_vertexai import ChatVertexAI from langchain.chains import ConversationalRetrievalChain from langchain_core.prompts import PromptTemplate from langchain.memory import ConversationBufferMemory # Set environment variables (Google API keys, etc.) os.environ["GOOGLE_API_KEY"] = 'YOUR_GOOGLE_API_KEY' os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "/path/to/your/credentials.json" # Initialize models and embeddings llm = ChatVertexAI(model="gemini-1.5-flash") embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") # Load, chunk, and index documents loader = TextLoader('Data Collection/corpus_combined.txt') docs = loader.load() text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) splits = text_splitter.split_documents(docs) vectorstore = Chroma.from_documents(documents=splits, embedding=embeddings, persist_directory='./test') retriever = vectorstore.as_retriever() # Define the prompt template template = """Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. If the question has a sense of urgency, prioritize answering with clarity and in maximum 3 sentences. If the question is more general, provide a more detailed explanation in 5-7 sentences. If you do not know the answer, you can provide a general response from what you know. Always say "thanks for asking!" at the end of the answer. {context} Question: {question} Helpful Answer:""" custom_rag_prompt = PromptTemplate.from_template(template) # Create memory object for chat history memory = ConversationBufferMemory( memory_key="chat_history", return_messages=True ) # Create ConversationalRetrievalChain qa_chain = ConversationalRetrievalChain.from_llm( llm=llm, retriever=retriever, memory=memory, combine_docs_chain_kwargs={"prompt": custom_rag_prompt} ) def get_answer(question): result = qa_chain({"question": question}) return result['answer'] ``` ### HikerAI/Model/HikerDetailItem.swift ```swift import Foundation // Conforming to Identifiable and Equatable struct HikerDetailItem: Identifiable, Equatable { let id = UUID() // Unique identifier let name: String // The item name } ``` ### HikerAI/View/LoadingIndicator.swift ```swift //import SwiftUI // //struct LoadingIndicator: View { // var body: some View { // ProgressView("Thinking...") // .progressViewStyle(CircularProgressViewStyle()) // .padding() //// } //} ``` ### HikerAI/Model/EmergencyContact.swift ```swift // // EmergencyContact.swift // HikerAI // // Created by Arnur Sabet on 20.10.2024. // import Foundation struct EmergencyContact: Identifiable { let id = UUID() var name: String var phone: String var relation: String } ``` [20 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]