# Project export: Oasis This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: AI agents help benchmark, assess, and enhance corporate sustainability practices - Devpost: https://devpost.com/software/oasis-8iv1lu - GitHub: https://github.com/anishk23733/oasis-scripts - Demo: https://github.com/anishk23733/oasis-backend - Video: https://www.youtube.com/embed/WqIzBEMv1_Q?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Endex: Senior Citizen Prize ($1k Cash)) - Team: 1 GitHub contributor(s) — Anish Kachinthaya (12 commits) ## Devpost submission (written by the team) ### Inspiration As the world grapples with challenges like climate change, resource depletion, and social inequality, it has become imperative for organizations to not only understand their environmental, social, and governance (ESG) impacts but also to benchmark and improve upon them. However, one of the most significant hurdles in this endeavor is the complexity and inaccessibility of sustainability data, which is often buried in lengthy official reports and varied formats, making it challenging for stakeholders to extract actionable insights. Recognizing the potential of AI to transform this landscape, we envision Oasis as a solution to democratize access to sustainability data, enabling more informed decision-making and fostering a culture of continuous improvement toward global sustainability goals. By conversing with AI agents, companies are able to collaborate in real-time to gain deeper insights and work towards solutions. ### What it does Oasis is a groundbreaking platform that leverages AI agents to streamline the parsing, indexing, and analysis of sustainability data from official government and corporate ESG reports. It provides an interface for companies to assess their records and converse with an AI agent that has access to their sustainability data. The agent helps them benchmark their practices against practices of similar companies and narrow down ways that they can improve through conversation. Companies can effortlessly benchmark their current sustainability practices, assess their current standings, and receive tailored suggestions for enhancing their sustainability efforts. Whether it's identifying areas for improvement, tracking progress over time, or comparing practices against industry standards, Oasis offers a comprehensive suite of features to empower organizations in their sustainability journey. ### How we built it Oasis uses a sophisticated blend of the following: LLM (LLaMA 2) parsing to parse data from complex reports. We fine-tuned an instance of meta-llama/Llama-2-7b-chat-hf on the HuggingFace dataset Government Report Summarization using MonsterAPI. We use this model to parse data points from ESG PDF text, since these documents are in a non-standard format, into a JSON format. LLMs are incredibly powerful at extracting key information and summarization, which is why we see such a strong use case here. LLM (LLaMA 2) parsing to parse data from complex reports. We fine-tuned an instance of meta-llama/Llama-2-7b-chat-hf on the HuggingFace dataset Government Report Summarization using MonsterAPI. We use this model to parse data points from ESG PDF text, since these documents are in a non-standard format, into a JSON format. LLMs are incredibly powerful at extracting key information and summarization, which is why we see such a strong use case here. Open-source text embedding model (SentenceTransformers) to index data including metrics and data points within a vector database. LLM-parsed data points contain key descriptors. We use an embedding model to index these descriptors in semantic space, allowing us to compare similar metrics across companies. Two key points may not have the same descriptions, but are semantically similar, which is why indexing with embeddings is beneficial. We use the SentenceTransformer model msmarco-bert-base-dot-v5 for text embeddings. We also use the InterSystems IRIS Data Platform to store embedding vectors, on top of the LangChain framework. This is useful for finding similar metrics across different companies and also for RAG, as discussed next. Open-source text embedding model (SentenceTransformers) to index data including metrics and data points within a vector database. LLM-parsed data points contain key descriptors. We use an embedding model to index these descriptors in semantic space, allowing us to compare similar metrics across companies. Two key points may not have the same descriptions, but are semantically similar, which is why indexing with embeddings is beneficial. We use the SentenceTransformer model msmarco-bert-base-dot-v5 for text embeddings. We also use the InterSystems IRIS Data Platform to store embedding vectors, on top of the LangChain framework. This is useful for finding similar metrics across different companies and also for RAG, as discussed next. Retrieval augmented generation (RAG) to incorporate relevant metrics and data points into conversation To enable users to converse with the agent and inspect and make decisions based on real data, we use RAG integrated with our IRIS vector database, running on the LangChain framework. We have a frontend UI for interacting with our agent in real time. Retrieval augmented generation (RAG) to incorporate relevant metrics and data points into conversation To enable users to converse with the agent and inspect and make decisions based on real data, we use RAG integrated with our IRIS vector database, running on the LangChain framework. We have a frontend UI for interacting with our agent in real time. Embedding similarity to semantically align data points for benchmarking across companies Our frontend UI also presents key metrics for benchmarking a user’s company. It uses embedding similarity to find company metrics and relevant metrics from other companies. Embedding similarity to semantically align data points for benchmarking across companies Our frontend UI also presents key metrics for benchmarking a user’s company. It uses embedding similarity to find company metrics and relevant metrics from other companies. ### Challenges we ran into One of the most challenging parts of the project was prompting the LLM and running numerous experiments until the LLM output matched what was expected. Since LLMs are non deterministic in nature and we required outputs in a consistent JSON form (for parsed results), we needed to prompt the LLM and reinforce the constraints multiple times. This was a valuable lesson that helped us learn how to leverage LLMs in intricate ways for niche applications. ### Accomplishments we're proud of We are incredibly proud of developing a platform that not only addresses a critical global challenge but does so with a level of sophistication and accessibility that sets a new standard in the field. Successfully training AI models to navigate the complexities of ESG reports marks a significant technical achievement. The ability to turn dense reports into clear, actionable insights represents a leap forward in sustainability practice. ### What we learned Throughout the process of building Oasis, we learned the importance of interdisciplinary collaboration in tackling complex problems. Combining AI and sustainability expertise was crucial in understanding both the technical and domain-specific challenges. We also gained insights into the practical applications of AI in real-world scenarios, particularly in how NLP and machine learning can be leveraged to extract and analyze data from unstructured sources. The iterative process of testing and feedback was invaluable, teaching us that user experience is as important as the underlying technology in creating impactful solutions. ### What's next The journey for Oasis is just beginning. Our next steps involve expanding the corpus of sustainability reports to cover a broader range of industries and geographies, enhancing the platform's global applicability. We are also exploring the integration of predictive analytics to offer forward-looking insights, enabling users to not just assess their current practices but also to anticipate future trends and challenges. Collaborating with sustainability experts and organizations will remain a priority, as their insights will help refine our models and ensure that Oasis continues to meet the evolving needs of its users. Ultimately, we aim to make Oasis a cornerstone in the global effort towards more sustainable practices, driving change through data-driven insights and recommendations. ## README (from the GitHub repository) # Data Orchestration for Oasis at TreeHacks 2024 ### Setup ``` python -m venv venv source venv/bin/activate pip install -r requirements.txt python -m spacy download en_core_web_sm ``` Create a `.env` file with your `TOGETHER_API_KEY` from https://www.together.ai/, `MONSTER_API_KEY` from https://monsterapi.ai/, and `PREDICTIONGUARD_TOKEN` from Intel Developer Cloud. If you are using a custom Monster Deployment, which we are, include the `MONSTER_DEPLOY_API_KEY` and `MONSTER_DEPLOY_URL` in the `.env` as well. ### Pipeline First run `scraper.py` to build a `database.json` and pull documents. Then run `cache_pdf.py` to pull text data from the scraped documents and cache them. Then run `llm_page_parse.py` to run LLaMA 2 on these documents and extract key metrics for Oasis. Then run `clean.py` to clean up the keyword data and save it in a more logical format. Then run `populate_vectordb.py` to populate the vector db with embedded keyword data. You can use the notebook `query_vectordb.ipynb` to inspect the data. You can also use the notebook `rag.py` to converse with the agent and gain insights about a company's sustainability practices. ### Set up VectorDB ``` git clone git@github.com:alvin-isc/treehacks-2024.git intersystems cd intersystems docker run -d --name iris-comm -p 1972:1972 -p 52773:52773 intersystemsdc/iris-community:2024.1-preview pip install -r requirements.txt docker run -d -p 27017:27017 --name test-mongo mongo:latest ``` Follow other instructions at https://github.com/alvin-isc/treehacks-2024/blob/main/demo/langchain_demo.ipynb to get the IRIS password set up. ## Detected evidence (automated analysis) Indexed codebase: 12 recognized source files, 25 KB. - LangChain (technology) — detected in the code - Python (language) — detected in the code - Flask (technology) — claimed on Devpost, not found in the code - MongoDB (technology) — claimed on Devpost, not found in the code - Next.js (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (15 of 15) ``` .gitignore cache_pdf.py clean.py llm_page_parse.py llm/intel.py llm/monster.py llm/monsterdeployment.py llm/togetherai.py populate_vectordb.py query_vectordb.ipynb rag.py README.md requirements.txt sandbox.py scraper.py ``` ### Dependencies - requirements.txt: BeautifulSoup4, langchain, langchain-together, monsterapi@==1.0.2b3, predictionguard, pymongo, pypdf2, python-dotenv, spacy ### Recent commits (newest first) - added mongodb support - added populating of mongodb database - fixed more rag - fixes - Update README.md - completed rag bot - working on rag now - added intersystems - updates - added a new llm - added llm support and parsing - initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` pypdf2 BeautifulSoup4 spacy langchain python-dotenv predictionguard langchain-together monsterapi==1.0.2b3 pymongo ``` ### sandbox.py ```python import os import json from llm.monsterdeployment import generate_text as generate_text_monsterd from llm.togetherai import generate_text as generate_text_togetherai generate_text = generate_text_togetherai with open("database.json", 'r') as f: data = json.load(f) def generate_prompt(page_data): return """ """ + str(page_data) + "\nend``` \n ## Output\n" for company in data.keys(): for year in data[company]: print(company, year) cleaned_path = os.path.join('cleaned', company, year + '.json') with open(cleaned_path, 'r') as f: cleaned_data = json.load(f) # government metrics x = list(filter(lambda x: x["topic"] == "G" and x["metric"], cleaned_data)) for item in x: print(item) # print(generate_text(generate_prompt(x))) # # government key points # for item in (filter(lambda x: x["topic"] == "G" and not x["metric"], cleaned_data)): # print(item) # # social metrics # for item in (filter(lambda x: x["topic"] == "S" and x["metric"], cleaned_data)): # print(item) # # social key points # for item in (filter(lambda x: x["topic"] == "S" and not x["metric"], cleaned_data)): # print(item) # # environmental metrics # for item in (filter(lambda x: x["topic"] == "E" and x["metric"], cleaned_data)): # print(item) # # environmental key points # for item in (filter(lambda x: x["topic"] == "E" and not x["metric"], cleaned_data)): # print(item) exit() ``` ### cache_pdf.py ```python from PyPDF2 import PdfReader import os import json import spacy import re nlp = spacy.load("en_core_web_sm") if not os.path.exists('text'): os.mkdir('text') def get_pdf_text(pdf_doc): pdf_reader = PdfReader(pdf_doc) for page in pdf_reader.pages: text = page.extract_text() yield text with open("database.json", 'r') as f: data = json.load(f) def check_coherence(sentence): has_subject = False has_verb = False for token in sentence: if "subj" in token.dep_: has_subject = True if "VERB" == token.pos_: has_verb = True # If any sentence lacks a subject or a verb, return False if not (has_subject and has_verb): return False # If all sentences seem coherent, return True return True for company in data.keys(): for item in data[company]: print(company, item) path = os.path.join('data', company, item + '.pdf') json_path = item + '.json' parent_dir = os.path.join('text', company) # This report has already been parsed if os.path.exists(os.path.join(parent_dir, json_path)): continue if not os.path.exists(parent_dir): os.mkdir(parent_dir) page_data = { 'pages': [] } for page in get_pdf_text(path): text = re.sub(r'\n+', ' ', page) doc = nlp(text) sentences = [] for sentence in doc.sents: if check_coherence(sentence): sentences.append(str(sentence)) if sentences: page_data['pages'].append(sentences) with open(os.path.join(parent_dir, json_path), 'w') as f: json.dump(page_data, f) ``` ### clean.py ```python import os import json import tqdm if not os.path.exists('cleaned'): os.mkdir('cleaned') with open("database.json", 'r') as f: data = json.load(f) def find_substring(full_string, start_marker, end_marker): # Find the start of the desired substring start_index = full_string.find(start_marker) if start_index == -1: return None # Start marker not found # Adjust start_index to get the actual beginning of the substring start_index += len(start_marker) # Find the end of the desired substring end_index = full_string.find(end_marker, start_index) if end_index == -1: return None # End marker not found # Extract and return the substring return full_string[start_index:end_index] for company in data.keys(): for year in data[company]: print(company, year) path = os.path.join('parsed', company, year + '.json') cleaned_path = os.path.join('cleaned', company, year + '.json') if os.path.exists(cleaned_path): continue parent_dir = os.path.join('cleaned', company) parse_data = [] if not os.path.exists(parent_dir): os.mkdir(parent_dir) with open(path, 'r') as f: llm_data = json.load(f) i = 0 for page in llm_data['parsed_pages']: if type(page) == list: combined = [] for sub_page in page: sub_page = find_substring(sub_page, '```start', 'end```') try: parsed = json.loads(sub_page) for val in parsed: val["id"] = f"{year}.{i}" i += 1 combined += parsed except: pass parse_data += combined else: sub_page = find_substring(page, '```start', 'end```') try: parsed = json.loads(sub_page) for val in parsed: val["id"] = f"{year}.{i}" i += 1 parse_data += parsed except: pass with open(cleaned_path, 'w') as f: json.dump(parse_data, f) ``` ### scraper.py ```python import requests from bs4 import BeautifulSoup import os import re import json companies = ['nvidia', 'apple', 'microsoft', 'tesla', 'intel', 'qualcomm', 'advanced micro devices', 'Taiwan Semiconductor Manufacturing'] OLDEST_YEAR = 2019 search_url = lambda x: f"https://www.responsibilityreports.com/Companies?search={x}" make_url = lambda x: f"https://www.responsibilityreports.com{x}" data = {} if os.path.exists('database.json'): with open("database.json", 'r') as f: data = json.load(f) def get_company_url(company): if not os.path.exists('data'): os.mkdir('data') response = requests.get(search_url(company)) html_content = response.text soup = BeautifulSoup(html_content, 'html.parser') first_company = soup.find('span', class_='companyName').find('a') company_name = first_company.text if company_name not in data: data[company_name] = [] else: # If company is already done, don't do it return company_url = make_url(first_company['href']) response = requests.get(company_url) html_content = response.text soup = BeautifulSoup(html_content, 'html.parser') parent_dir = f'data/{company_name}' if not os.path.exists(parent_dir): os.mkdir(parent_dir) try: most_recent = soup.find('div', class_='most_recent_content_block').find('div', class_='view_btn').find('a') report_url = make_url(most_recent['href']) response = requests.get(report_url) year = most_recent.text[:4] file_name = f"{year}.pdf" data[company_name].append(most_recent.text[:4]) with open(os.path.join(parent_dir, file_name), 'wb') as f: f.write(response.content) except: print("Latest report failed.") spans = soup.find_all('span', class_='btn_archived download') for span in spans: link = span.find('a')['href'] response = requests.get(make_url(link)) try: title = span.parent.find('span', class_="heading").text year = str(title)[:4] if int(year) < OLDEST_YEAR: return file_name = f"{year}.pdf" data[company_name].append(year) with open(os.path.join(parent_dir, file_name), 'wb') as f: f.write(response.content) except: print("Older report failed.") for company in companies: get_company_url(company) with open("database.json", 'w') as f: json.dump(data, f) with open("database.json", 'w') as f: json.dump(data, f) ``` ### populate_vectordb.py ```python import os from dotenv import load_dotenv import json import time import tqdm load_dotenv() from langchain.docstore.document import Document from langchain_together import TogetherEmbeddings from langchain_iris import IRISVector from sqlalchemy.exc import OperationalError from langchain.embeddings import HuggingFaceEmbeddings from pymongo import MongoClient # embeddings = TogetherEmbeddings(model="togethercomputer/m2-bert-80M-2k-retrieval") embeddings = HuggingFaceEmbeddings(model_name="all-mpnet-base-v2") client = MongoClient() username = 'SUPERUSER' password = 'oasis' # Replace password with password you set # http://localhost:52773/csp/sys/UtilHome.csp # change from 'SYS' hostname = 'localhost' port = '1972' namespace = 'USER' CONNECTION_STRING = f"iris://{username}:{password}@{hostname}:{port}/{namespace}" COLLECTION_NAME = "vectordb" db = IRISVector( embedding_function=embeddings, dimension=768, collection_name=COLLECTION_NAME, connection_string=CONNECTION_STRING, ) db.delete_collection() db.create_collection() client.vectordb.drop_collection("company_key_data") with open("database.json", 'r') as f: data = json.load(f) BATCH_SIZE=1 for company in data.keys(): for year in data[company]: print(company, year) cleaned_path = os.path.join('cleaned', company, year + '.json') with open(cleaned_path, 'r') as f: cleaned_data = json.load(f) docs = [] entries = [] for entry in tqdm.tqdm(cleaned_data): entry['year'] = year entry['company'] = company entries.append(entry) # Upload entry to MongoDB docs.append(Document(page_content=entry["description"], metadata=entry)) if len(docs) > BATCH_SIZE: # Upload batch to IRIS uploaded = False while not uploaded: try: db.add_documents(docs) uploaded = True except (OperationalError): time.sleep(2) docs = [] # Upload batch to MongoDB if entries: client.vectordb.company_key_data.insert_many(entries) entries = [] # Upload batch to IRIS uploaded = False while not uploaded: try: db.add_documents(docs) uploaded = True except (OperationalError): time.sleep(2) # Upload batch to MongoDB if entries: client.vectordb.company_key_data.insert_many(entries) time.sleep(.1) # break ``` ### rag.py ```python from langchain_iris import IRISVector from langchain.embeddings import HuggingFaceEmbeddings from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser from langchain_core.runnables import RunnablePassthrough from langchain_together import Together from dotenv import load_dotenv class RAG(): def __init__(self, company, model): embeddings = HuggingFaceEmbeddings(model_name="all-mpnet-base-v2") username = 'SUPERUSER' password = 'oasis' hostname = 'localhost' port = '1972' namespace = 'USER' CONNECTION_STRING = f"iris://{username}:{password}@{hostname}:{port}/{namespace}" COLLECTION_NAME = "vectordb" self.db = IRISVector( embedding_function=embeddings, dimension=768, collection_name=COLLECTION_NAME, connection_string=CONNECTION_STRING) self.retriever = self.db.as_retriever(search_type="similarity_score_threshold", search_kwargs={ "k": 50, "score_threshold": .15, "filter": {} }) self.model = model self.company = company template = """ [INST] You are an agent speaking with a representative from {company}. Your goal is to assist the representative with questions regarding their sustainability practices. You are provided context regarding {company}'s sustainablity practices and your goal is to compare their practices with practices of other companies, for which context is given to you, in order to help them improve. Be critical with your response as well by noticing when metrics are missing or where they could be better. Answer the question with a detailed response based only on the following context, incorporating data from other companies: {context} Conversation History: {history} Representative: {question} [/INST] """ self.prompt = ChatPromptTemplate.from_template(template) # Initialize conversation history self.conversation_history = "" def update_history(self, question, answer): self.conversation_history += f"Representative: {question}\nAgent: {answer}\n" def get_response(self, input_query): # Define the chain with updated conversation history in the context chain = ( { "context": self.retriever, "history": lambda x: self.conversation_history, "company": lambda x: self.company, "question": RunnablePassthrough() } | self.prompt | self.model | StrOutputParser() ) output = chain.invoke(input_query) self.update_history(input_query, output) return output def chat_interface(self): while True: user = input(">>> ") if user == "stop": break print(self.get_response(user)) if __name__ == "__main__": company = "NVIDIA Corporation" load_dotenv() model = Together( model="mistralai/Mixtral-8x7B-Instruct-v0.1", temperature=0.7, max_tokens=1024, top_k=50, ) r = RAG(company, model) r.chat_interface() ``` ### llm_page_parse.py ```python from llm.togetherai import generate_text as generate_text_togetherai from llm.intel import generate_text as generate_text_intel from llm.monster import generate_text as generate_text_monster import os import json import tqdm import threading import time generate_text = generate_text_monster # run both intel and monster generate_text = generate_text_intel generate_text = generate_text_togetherai def generate_prompt(page_data): return """ You are an analyst extracting key metrics and points from a page in an official document.\n Note that you do not care about redundant information or irrelevant points. You only extract key points that provide useful insights about a company's CSR. The output should be a list of objects, where each object represents a single point or fact, and contains the following properties: * value: The value of the point or fact, which can be a number, a metric, or a descriptive text. * metric: A boolean indicating whether the value is a metric or not. * topic: The topic of the point or fact, which can be "E" for environmental, "S" for social, or "G" for governance. * description: A brief description of the point or fact, providing context and additional information. * tags: An array of keywords or tags associated with the point or fact, which can be used for filtering or searching. Do not include general statements. Only include points mentioning organizations, statistics, or proven action. You respond with concise and accurate JSON formatted output, like as follows:\n ## Input\n ```start [ "Renewable electricity Our retail stores, data centers, and offices around the world currently source 100 percent renewable electricity.", "Over 70 percent of companies on Apple\u2019s Supplier List \u2014 those suppliers that make up 98 percent of Apple\u2019s direct spend for materials, manufacturing, and assembly of our products worldwide \u2014 have committed to 100 percent renewable electricity.", "In addition, many other smaller suppliers have also made these commitments.", "About 1.5 gigawatts of Apple-created renewable electricity projects account for over 90 percent of the renewable electricity our facilities use.", "In fiscal year 2021, Apple avoided 180,000 metric tons of CO 2e by shifting the mode of transport and reducing product weight through the removal of the power adapter from the box of iPhone devices.", "And we\u2019ve expanded our relationship with Bureau of Energy Resources, initiating new government contracts.", "We also offer our U.S. employees a transit subsidy of up to $100 per month, and at our Cupertino and surrounding Santa Clara Valley campus, we offer free coach buses to commute to and from our corporate offices.", "Apple has invested in the 2300-acre IP Radian Solar project in Brown County, Texas." ] end``` ## Output\n ```start [ { "value": "70 percent", "metric": true, "topic": "E", "description": "Percent of companies on Apple's Supplier List that have committed to 100 percent renewable electricity, making 98 percent of Apple's direct spend for materials, manufacturing, and assembly of products worldwide.", "tags": ["supplier", "renewable energy"] }, { "value": "1.5 gigawatts", "metric": true, "topic": "E", "description": "Created from renewable energy projects that account for over 90 percent of the renewable electricity our facilities use.", "tags": ["renewable energy"] }, { "value": "180,000 metric tons", "metric": true, "topic": "E", "description": "Metric tons of CO 2e avoided by shifting the mode of transport and reducing product weight through the removal of the power adapter from the box of iPhone devices.", "tags": ["carbon emissions"] }, { "value": "$100", "metric": true, "topic": "S", "description": "Monthly transit subsidy for employees.", "tags": ["employee benefits"] }, { "value": "Bureau of Energy Resources", "metric": false, "topic": "G", "description": "Expansion of relationship with Bureau of Energy Resources with new government contracts.", "tags": ["partnerships"] }, { "value": "none", "metric": false, "topic": "S", "description": "Offer free coach buses to commute to and from corporate offices at Cupertino and surrounding the Santa Clara Valley campus.", "tags": ["employee benefits"] }, { "value": "IP Radian Solar project", "metric": false, "topic": "E", "description": "Offer free coach buses to commute to and from corporate offices at Cupertino and surrounding the Santa Clara Valley campus.", "tags": ["renewable energy"] } ] end``` Note that topic of the point or fact can only be "E", "S", or "G". Provide analysis output for the following data, as a JSON list:\n ## Input\n ```start""" + str(page_data) + "end```## Output\n" if not os.path.exists('parsed'): os.mkdir('parsed') with open("database.json", 'r') as f: data = json.load(f) for company in data.keys(): for year in data[company]: print(company, year) path = os.path.join('text', company, year + '.json') parsed_path = os.path.join('parsed', company, year + '.json') if os.path.exists(parsed_path): continue parent_dir = os.path.join('parsed', company) parse_data = [] if not os.path.exists(parent_dir): os.mkdir(parent_dir) with open(path, 'r') as f: page_data = json.load(f) def threading_task(page): try: result = generate_text(generate_prompt(page)) parse_data.append(result) except Exception as e: try: time.sleep(1.1) result0 = generate_te [truncated — 836 more characters] ``` ### llm/intel.py ```python import predictionguard as pg from dotenv import load_dotenv import json load_dotenv() def generate_text(prompt, system_prompt=""): response = pg.Completion.create(model="Llama-2-7B", prompt=prompt, max_tokens=1024) return response['choices'][0]['text'] ``` ### llm/togetherai.py ```python from langchain_together import Together import os from dotenv import load_dotenv load_dotenv() llm = Together( # model="deepseek-ai/deepseek-coder-33b-instruct", model="meta-llama/Llama-2-70b-chat-hf", temperature=0.7, max_tokens=1028, top_k=1, ) def generate_text(prompt, system_prompt=""): return llm.invoke(system_prompt + '\n' + prompt) ``` [2 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]