# Project export: WeHeal This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: An AI powered by Langchain to assist counselors summarize patient case files and offer personalized suggestions based on past interactions, simplifying workflow and enhancing session effectiveness. - Devpost: https://devpost.com/software/weheal - GitHub: https://github.com/AshishAgarwal2101/TreeHacks2024 - Video: https://www.youtube.com/embed/1KA1iihK3-4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 3 GitHub contributor(s) — Ashish Agarwal (22 commits), Manikanta Sanjay Veera (3 commits), MridangK (1 commits) ## Devpost submission (written by the team) ### Inspiration Our inspiration for WeHeal stemmed from a deep-seated desire to revolutionize the counseling and therapy landscape, aiming to enhance both efficiency and effectiveness. Recognizing the challenge faced by counselors and therapists in managing extensive patient case files while striving to deliver personalized care, we embarked on a journey to create a solution. Consider a scenario where a counselor, with a packed schedule of five patients per day across four days a week, grapples with recalling every detail of a patient's conversation, especially as sessions progress. We sought to bridge this gap by developing a tool that seamlessly integrates with practitioners' workflows, empowering them to provide unparalleled support and care to their clients, regardless of session volume or complexity. ### What it does WeHeal serves as an invaluable AI assistant powered by LangChain to counselors and therapists, streamlining their workflow by efficiently summarizing patient case files and providing tailored recommendations derived from both past and ongoing interactions. Our platform enhances the effectiveness and efficiency of counseling sessions, while keeping the past context to ultimately empower professionals to better support their clients on their journey towards healing and growth. ### How we built it ---The project is divided mainly into three parts: Agents powered by Intel's "Prediction Guard" LLM models: to store user data into vector database, contextuatize the user data. The chat history is stored in a Momento DB cache. Agents support the core logic of our application, including summarization of user details based on PDF case data and to provide tailored suggestions to the counselors. Frontend: Built in ReactJS uses the APIs exposed by the backend to showcase how the power of our AI companion. It uses "Canva.dev" for few components. API Backend: Built with flask to bridge the gap between the Agent and the Frontend. ---The deployment is done using "Vercel", as shown in the Demo. ---We built WeHeal using a combination of programming languages and technologies, including Python for backend development, React for the frontend, and various APIs for data analysis and natural language processing. We also utilized cloud services for hosting and deployment. ### Challenges we ran into None of us knew how to do front end development so, we had to learn and deploy at the same time, Canva.dev was a huge help for us in this challenge. Another issue that we faced was LanceDB, which is apparently new to the market with limited community support. Majority issues were faced with LLM chaining together to provide context with chat history and user details. Integrating different components together was also a challenge towards the end. ### Accomplishments we're proud of We ar proud of our idea which could a game-changer for the superheroes who are the counselors helping patients fight mental health issues. We're proud to have successfully developed a functional end to end prototype of WeHeal within the timeframe of the hackathon. This prototype can succesfully be used to demonstrate the power of our LLM agents. Our platform demonstrates the potential to significantly improve the workflow of counselors and therapists, ultimately benefiting both professionals and their clients. ### What we learned Through the process of building WeHeal, we gained valuable experience in full stack web development, RAG, VectorDB, Langchain, and many more technologies that were completely new to us. We also learnt the existence of amazing companies oproviding amazing solutions, some of which we integrated in our application. This includes Intel's Prediction Guard, Canva.dev components, and deployments via VercelW ### What's next Looking ahead, our vision for WeHeal involves a comprehensive evolution aimed at empowering users on their mental wellness journey. We're committed to refining and enhancing the platform to offer a personalized space where individuals can seamlessly upload their daily thoughts. Leveraging advanced algorithms and user data, WeHeal will then intelligently suggest affirmations and curated meditation resources tailored to each user's unique needs and emotional state. Moreover, we're exploring innovative solutions such as blockchain technology to securely store patient case files. This ensures that individuals can seamlessly transition between therapists or counselors without any loss of crucial information. By implementing blockchain, we guarantee the integrity and accessibility of patient records, enabling new practitioners to gain comprehensive insights through smart summaries, thus fostering continuity of care and empowering individuals to receive the support they need, wherever they may be. ## README (from the GitHub repository) # TreeHacks2024 # WeHeal ## Introduction Welcome to WeHeal, a revolutionary platform designed to transform the counseling and therapy landscape. Our goal is to enhance the efficiency and effectiveness of counseling sessions by providing counselors and therapists with powerful AI-driven tools. With WeHeal, professionals can streamline their workflows, gain valuable insights from patient interactions, and ultimately deliver personalized care to their clients. ## Inspiration WeHeal was born out of a profound desire to address the challenges faced by counselors and therapists in managing extensive patient case files while striving to deliver personalized care. We envisioned a solution that seamlessly integrates into practitioners' workflows, empowering them to provide unparalleled support and care to their clients, regardless of session volume or complexity. ## What it Does WeHeal serves as an invaluable AI assistant powered by LangChain. It streamlines counselors' and therapists' workflows by efficiently summarizing patient case files and providing tailored recommendations derived from past and ongoing interactions. By retaining past context, WeHeal enables professionals to better support their clients on their journey towards healing and growth. ## How We Built It WeHeal is comprised of three main components: 1. **Agents powered by Intel's "Prediction Guard" LLM models:** These agents store user data into a vector database and contextualize the data. They support the core logic of our application, including summarizing user details based on PDF case data and providing tailored suggestions to counselors. 2. **Frontend:** Built in ReactJS, the frontend utilizes APIs exposed by the backend to showcase the power of our AI companion. Canva.dev was used for some components. 3. **API Backend:** Built with Flask, the backend bridges the gap between the Agent and the Frontend. Deployment is done using Vercel, as demonstrated in the demo. We utilized a combination of programming languages and technologies, including Python for backend development, React for the frontend, and various APIs for data analysis and natural language processing. Cloud services were used for hosting and deployment. ## Challenges We Ran Into One major challenge was the lack of expertise in frontend development, which required us to learn and deploy simultaneously. Canva.dev proved to be immensely helpful in overcoming this challenge. Additionally, integrating LanceDB, a relatively new database technology, presented challenges due to limited community support. We also faced difficulties with LLM chaining to provide context with chat history and user details. ## Accomplishments We're Proud Of We're proud to have developed a functional end-to-end prototype of WeHeal within the timeframe of the hackathon. Our platform demonstrates the potential to significantly improve the workflow of counselors and therapists, benefiting both professionals and their clients. We believe WeHeal could be a game-changer for counselors helping patients fight mental health issues. ## What We Learned Building WeHeal provided us with valuable experience in full-stack web development, RAG, VectorDB, LangChain, and various other technologies that were previously unfamiliar to us. We also discovered amazing companies providing solutions that we integrated into our application, including Intel's Prediction Guard, Canva.dev components, and deployment via Vercel. ## What's Next for WeHeal Moving forward, we envision a comprehensive evolution of WeHeal aimed at empowering users on their mental wellness journey. We're committed to refining and enhancing the platform to offer a personalized space where individuals can seamlessly upload their daily thoughts. Leveraging advanced algorithms and user data, WeHeal will intelligently suggest affirmations and curated meditation resources tailored to each user's unique needs and emotional state. Furthermore, we're exploring innovative solutions such as blockchain technology to securely store patient case files. This ensures seamless transitions between therapists or counselors without any loss of crucial information. By implementing blockchain, we guarantee the integrity and accessibility of patient records, fostering continuity of care and empowering individuals to receive the support they need, wherever they may be. ## Demo To see WeHeal in action, check out our demo [here](https://youtu.be/1KA1iihK3-4?si=R3pFc6LXDz4OApKn). ## Contributors * Manikanta Sanjay Veera * Ashish Agarwal * Mridang Kejriwal ## Detected evidence (automated analysis) Indexed codebase: 13 recognized source files, 34 KB. - CSS (language) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - LangChain (technology) — detected in the code - LlamaIndex (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (24 of 24) ``` .DS_Store .gitignore backend/.gitignore backend/index.py backend/RAG_updated.ipynb backend/requirements.txt backend/vercel.json frontend/.eslintrc.cjs frontend/.gitignore frontend/index.html frontend/package.json frontend/README.md frontend/src/ApiCalls.tsx frontend/src/App.css frontend/src/App.tsx frontend/src/Home.tsx frontend/src/index.css frontend/src/main.tsx frontend/src/UserEntry.tsx frontend/src/vite-env.d.ts frontend/tsconfig.json frontend/tsconfig.node.json frontend/vite.config.ts README.md ``` ### Dependencies - backend/requirements.txt: flask, flask-cors, lancedb, langchain, llama-index, llama-parse, momento, predictionguard, sentence-transformers - frontend/package.json: @canva/app-ui-kit@^3.2.0, @types/node@^20.11.19, @types/react@^18.2.55, @types/react-dom@^18.2.19, @typescript-eslint/eslint-plugin@^6.21.0, @typescript-eslint/parser@^6.21.0, @vitejs/plugin-react@^4.2.1, bootstrap@^5.3.2, eslint@^8.56.0, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.5, react@^18.2.0, react-bootstrap@^2.10.1, react-dom@^18.2.0, react-router-dom@^6.22.1, typescript@^5.2.2, vite@^5.1.0 ### Recent commits (newest first) - Update README.md - URL - URL - URL - URL - URL - URL - URL - URL - Multiline input - URL - URL - Create vercel.json - add-backend-py - Bug - Typescript strict ignore - Bug resolution - Frontend main end to end - Frotend UI changes - Merge branch 'main' of https://github.com/AshishAgarwal2101/TreeHacks2024 ## Key source files (fetched from GitHub, selected and truncated for size) ### backend/requirements.txt ``` langchain lancedb llama-parse llama-index predictionguard sentence-transformers momento flask flask-cors ``` ### frontend/package.json ``` { "name": "treehacks", "private": true, "version": "0.0.0", "type": "module", "scripts": { "dev": "vite", "build": "tsc && vite build", "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0", "preview": "vite preview" }, "dependencies": { "@canva/app-ui-kit": "^3.2.0", "@types/node": "^20.11.19", "bootstrap": "^5.3.2", "react": "^18.2.0", "react-bootstrap": "^2.10.1", "react-dom": "^18.2.0", "react-router-dom": "^6.22.1" }, "devDependencies": { "@types/react": "^18.2.55", "@types/react-dom": "^18.2.19", "@typescript-eslint/eslint-plugin": "^6.21.0", "@typescript-eslint/parser": "^6.21.0", "@vitejs/plugin-react": "^4.2.1", "eslint": "^8.56.0", "eslint-plugin-react-hooks": "^4.6.0", "eslint-plugin-react-refresh": "^0.4.5", "typescript": "^5.2.2", "vite": "^5.1.0" } } ``` ### backend/index.py ```python import sys import os from langchain.chains import ConversationChain from langchain.memory import ConversationBufferMemory from langchain.schema import SystemMessage from langchain.llms import PredictionGuard from langchain.memory import ChatMessageHistory import os import requests os.environ['PREDICTIONGUARD_TOKEN'] = "q1VuOjnffJ3NO2oFN8Q9m8vghYc84ld13jaqdF7E" os.environ['MOMENTO_API_KEY'] = "eyJlbmRwb2ludCI6ImNlbGwtNC11cy13ZXN0LTItMS5wcm9kLmEubW9tZW50b2hxLmNvbSIsImFwaV9rZXkiOiJleUpoYkdjaU9pSklVekkxTmlKOS5leUp6ZFdJaU9pSnRZVzVwYTJGdWRHRnpZVzVxWVhreE9UazVRR2R0WVdsc0xtTnZiU0lzSW5abGNpSTZNU3dpY0NJNklrTkJRVDBpTENKbGVIQWlPakUzTVRBNE1ESTJPVGg5LjVLZm9HX1JGck02dmZqcTRQUU5QV0lVdXVFaHVZb2JBRWhSell4X1V0WGcifQ==" import nest_asyncio nest_asyncio.apply() from llama_parse import LlamaParse parser = LlamaParse( api_key="llx-VqWwj6hJue36ZZ5UhU0hufc1ArT6p3L3vqXI3zgQfH3jpzzs", # can also be set in your env as LLAMA_CLOUD_API_KEY result_type="markdown", # "markdown" and "text" are available verbose=True ) def load_document(path): documents = parser.load_data(path) text = documents[0].text # Clean things up just a bit. text = text.split("### I. PERSONAL INFORMATION")[1] return text from langchain.text_splitter import CharacterTextSplitter from sentence_transformers import SentenceTransformer import pandas as pd import lancedb from lancedb.embeddings import with_embeddings model = SentenceTransformer("all-MiniLM-L12-v2") def embed_batch(batch): res = [model.encode(sentence) for sentence in batch] return res def embed(sentence): return model.encode(sentence) def data_cleanup_and_formation(data, user_id): if(len(data) == 0): return pd.DataFrame() text_splitter = CharacterTextSplitter(chunk_size=700, chunk_overlap=50, separator="##") docs = text_splitter.split_text(data) docs = [x.replace('#', '-') for x in docs] docs = [x.replace('&emsp', '') for x in docs] metadata = [] for i in range(len(docs)): metadata.append([ user_id, i, docs[i] ]) doc_df = pd.DataFrame(metadata, columns=["user_id", "chunk", "text"]) # Embed the documents data = with_embeddings(embed_batch, doc_df) print("data: ", type(data)) return data ##### Lance DB setup uri = ".lancedbq" db = lancedb.connect(uri) import pyarrow as pa def save_user_data(user_id, text): data = data_cleanup_and_formation(text, user_id) # Create the DB table and add the records. try: db.create_table("counsellor", data) except Exception as e: print("Table counsellor does not exists") table = db.open_table("counsellor") #print("my table: ", type(table)) table.add(data=data) return data def retrieve_embeddings(user_id, message): table = db.open_table("counsellor") results = table.search(embed(message)).limit(2).to_df() return results from datetime import timedelta from langchain.memory import MomentoChatMessageHistory cache_name = "langchain" ttl = timedelta(days=1) import predictionguard as pg from langchain import PromptTemplate llm=PredictionGuard(model="Neural-Chat-7B") # Now let's augment our Q&A prompt with this external knowledge on-the-fly!!! template_start = """ Current conversation: {history} ### Instruction: Assume you are a mental health professional with 10+ years experience. You will get chat based questions from a patient. Based on the question, you must generate responses for the user. Make absolutely sure all your responses seem like actual responses from a mental health expert. Do not suggest the patient to seek alternatives or to seek other counsellors or therapists or mental health The input and output formats are defined below. Only reply based on the output format (replace with your response). """ template_summary = """ ### Input: Context: {context} Question: {question} ### Output: """ template_chat = """ Current conversation: {history} The input and output formats are defined below. Only reply based on the output format (replace with your response). ### Input: Context: {context} Question: {question} ### Output: """ users_history = {} users_chain = {} def init_user_history(user_id): instruction = '''Assume you are a mental health professional with 10+ years experience. You will get chat based questions from a patient. Based on the question, you must generate responses for the user. Make absolutely sure all your responses seem like actual responses from a mental health expert. Do not suggest the patient to seek alternatives or to seek other counsellors or therapists or mental health The input and output formats are defined below. Only reply based on the output format (replace with your response).''' users_chain[user_id]=ConversationChain( llm=llm, verbose=True ) users_history[user_id] = MomentoChatMessageHistory.from_client_params( user_id, cache_name, ttl, ) if len(users_history[user_id].messages)==0: users_history[user_id].add_user_message(instruction) users_history[user_id].add_ai_message("") import math def getMessageHistory(user_id, context, message): len_needed = len(context) + len(message) remaining_len = min(3000 - len_needed, len(message)) return users_history[user_id].messages[-remaining_len:] def get_user_summary(user_id, user_text): instruction = '''Assume you are a mental health professional with 10+ years experience. You will get chat based questions from a patient. Based on the question, you must generate responses for the user. Make absolutely sure all your responses seem like actual responses from a mental health expert. Do not suggest the patient to seek alternatives or to seek other counsellors or therapists or mental health The input and output formats are defined below. Only reply based on the output format (replace with your response).''' question = "Based on the text below, summarize the patient's key points in 200 words: \n\n " + user_text qa_prompt = [truncated — 5606 more characters] ``` ### frontend/src/main.tsx ```typescript import ReactDOM from 'react-dom/client' import App from './App' import './index.css' import 'bootstrap/dist/css/bootstrap.css' ReactDOM.createRoot(document.getElementById('root')).render() ``` ### frontend/src/App.tsx ```typescript import { BrowserRouter as Router, Routes, Route } from 'react-router-dom'; import "@canva/app-ui-kit/styles.css"; import './App.css' import Home from './Home'; import UserEntry from './UserEntry'; import { AppUiProvider } from "@canva/app-ui-kit"; function App() { return ( } /> } /> ); } export default App; ``` ### frontend/vite.config.ts ```typescript import { defineConfig } from 'vite' import react from '@vitejs/plugin-react' // https://vitejs.dev/config/ export default defineConfig({ plugins: [react()], }) ``` ### frontend/index.html ```html CounsellorAssistant
``` ### frontend/src/vite-env.d.ts ```typescript /// ``` ### frontend/src/index.css ```css body { margin: 0; /* Remove default margin */ padding: 0; } ``` ### frontend/src/UserEntry.tsx ```typescript export default function UserEntry(){ return(
); } ``` [3 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]