# Project export: brOSKI
This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.
## Project metadata
- Hackathon: Cal Hacks 10.0
- Tagline: Knowledgeable and Friendly AI Cal Redditor - For Bears by Bears!
- Devpost: https://devpost.com/software/broski
- GitHub: https://github.com/m0zzaR/CalCompanion
- Team: 3 GitHub contributor(s) — m0zzaR (5 commits), LeonShams (4 commits), pranavdo (1 commits)
## Devpost submission (written by the team)
### Inspiration
In the midst of Cal Enrollment and Berkeley college life in general, getting advice on what classes we should take, what spots in Berkeley to explore and what cafe's are the best, are sometimes hard information to come by. Thankfully, the Berkeley Subreddit has tons of Berkeley students just like us, who have their own unique ideas and opinions. The best way to represent all of this information is through language models! This is why we created brOSKI, a chat bot who can be your friend and advisor about all things Berkeley. How We Built it Scrape Reddit. With recent updates Reddit's API no longer permits data scraping, so collecting data from reddit on the scale needed required a unique approach. There is only one comprehensive archive of Reddit's database, "Pushshift." The problem is the archive has been removed from the public by Reddit, so to get the data from Pushshift we had to find an old copy of the database and download it in fragments. Bear in mind, the Pushshift Reddit database contains a recorded history of all of reddit, not just the Berkeley subreddit, meaning we had to grapple with over 20 terabytes of text data, to get the data we needed from the Berkeley subreddit. The smallest fragments the database can be downloaded in is 450GBs. By selectively only downloading posts, not comments we trimmed down the downloaded fragment size to 150GBs. We then filtered the downloaded posts to contain only r/Berkeley posts. Now we also needed the comments for each post, so we queried the API by post ID to collect the comments for the r/Berkeley posts we downloaded. Repeated a few times we managed to download months worth of r/Berkeley data (posts and comments) for usage by our LLM algorithms. Finetune LLM The next step in the process, is to take the scraped and formatted data fine-tune our model of choice. For this project, we decided to use LLaMA 2 13b. We first tested smaller data samples, using both a small amount of epochs and high amount to see the difference in model "sound". We noticed that the model when overfit to the small dataset, became biased and uncensored, answering questions it probably shouldn't. Once we had all our data scraped and formatted correctly, we passed the 3500 lines of Q&A style training data, and chose 2, 4, 6, 8, and 10 epochs. We discovered that mid to low epoch range seems to give the best results for our use case. Front End Initially, we planned to have an opening web page - quite similar to Chat GPT and bing AI with some example inputs along with few animations to make it seem more like iMessages or Instagram Direct Messages. However, due to time constraints, we are leaving that as future work. We designed our front end on Figma and primarily used HTML and CSS for our front end to put things in motion. At the end, we developed an API to connect our back end to the front end.
## README (from the GitHub repository)
# u/brOSKI
u/brOSKI is a chat bot that attempts to help Berkeley students with everything Cal while trained to sound like the average r/berkeley enjoyer. We accomplish
with a dual pronged approach. In order to get the "sound" of a funny berkeley student, I trained LLaMA 2 13b on the r/berkeley subreddit using the together.ai
API, and in order to get the knowledge and accuracy of a Cal advisor, LeonShams ustilized prompt engineering and LLama 2 13b to currate our data based on class
information, teacher biographies and enrollment information. This way our model could give helpful information to the user. pranavdo implemented the front end,
which displays a user interface that interacts with the backend to call LLama 2 12b.
u/brOSKI is finetuned with the help of together.ai, which offers a very easy to use API to train and deploy the model. Unfortunately, the API does not yet support
your finetuned models to be used with your local code. So unless you are signed into my account, you cannot YET talk to the finetunened u/brOSKI. Nonetheless,
there will be some pictures provided below of some of the funny answers u/brOSKI gave after chating with it for about 10 minutes.
This program was built during CalHacks 2023 so it has only been in development for about 48 hours (as of 10/29/2023). Considering the extremely
short time frame, we are extremely happy with the results of our model. All of the code that we used during those 48 hours are provided here
in this github. Stay tuned, because we have future visions of expanding the model even more!
# Here are some images of just the finetuned model, no contextual analysis is ran here.
*Finetuned chat bot, the "brO" side of u/brOSKI*
The data we used for this model was about 3500 lines of r/berkeley posts from 2023 formated in Q&A style. Where the title
and body are the "question" and the most upvoted comment is the "answer". The model is trained for 6 epochs. There
is MUCH MUCH room for improvement here considering our biggest limiting factor was finding and downloading data
within the CalHacks time frame.
-Note: The data that we curated removes personal user information.
# Guide for running context chat bot into a localHost*
*Context analysis chat bot, the "u/OSKI" side of u/brOSKI*
You need to use your own together.ai API Key to run this program.
Requirements:
```
pip install together
pip install flask
pip install flask_cors
```
Step 1> while inside ./CalCompanion run
```
python api.py
```
Step 2> open up the index.html fine
Step 3> Chat away!
## Detected evidence (automated analysis)
Indexed codebase: 13 recognized source files, 25 KB.
- HTML (language) — detected in the code
- Python (language) — detected in the code
- JavaScript (language) — claimed on Devpost, not found in the code
## Codebase structure (from repository index)
### Files (19 of 19)
```
.gitignore
contextual_inference/api.py
contextual_inference/broski_inference.py
contextual_inference/compressed_posts.txt
contextual_inference/threaded_chunk_processing.py
index.html
LICENSE
model_interaction/.gitattributes
model_interaction/data/data.rar
model_interaction/data/finalDataForCalHacks.jsonl
model_interaction/file_scraper/api_comments_from_posts.py
model_interaction/file_scraper/api_posts_from_comments.py
model_interaction/file_scraper/comment_extractor.py
model_interaction/file_scraper/file_export.py
model_interaction/file_scraper/submission_extractor.py
model_interaction/finetuning.py
model_interaction/parsing.py
model_interaction/prompting.py
README.md
```
### Dependencies
No dependency index available.
### Recent commits (newest first)
- Update README.md
- Update README.md
- Merge branch 'main' of https://github.com/m0zzaR/CalCompanion
- Final
- Update api_posts_from_comments.py
- Update broski_inference.py
- Update threaded_chunk_processing.py
- Update finetuning.py
- Update prompting.py
- Update finetuning.py
- Update api_comments_from_posts.py
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
- Update README.md
## Key source files (fetched from GitHub, selected and truncated for size)
### index.html
```html
Chatbot
brOSKI
Hello!
Hello I am u/broski! Your AI Berkeley redditor - made for Bears by Bears. Ask me anything!
```
### model_interaction/finetuning.py
```python
import sys
import together
together.api_key = ""
def preTrain():
resp = together.Files.upload(file="./finaloutput.jsonl")
return resp['id']
def train():
resp = together.Finetune.create(
training_file = preTrain(),
model = 'togethercomputer/llama-2-13b',
n_epochs = 2,
n_checkpoints = 1,
batch_size = 4,
learning_rate = 1e-5,
suffix = 'broski',
wandb_api_key = '',
)
return resp
```
### contextual_inference/api.py
```python
from flask import Flask, request, jsonify
from flask_cors import CORS # Import the CORS class
import together
from broski_inference import ask_question
app = Flask(__name__)
CORS(app)
together.api_key = "a66ca7b7091cd606df07d72d5f103a61a3f62762312437bcdf5304211e9f558e"
stream = ""
def processText(text):
global stream
stream += f": {text}\n:"
resp = ask_question(stream)
stream += resp
return resp
@app.route('/processText', methods=['POST'])
def handle_request():
data = request.get_json(force=True) # Get the request data as JSON
text = data.get('text', '') # Get the 'text' field from the request data
result = processText(text) # Call your function with the text
return jsonify(response=result) # Return the result as JSON
if __name__ == '__main__':
app.run(host='0.0.0.0', port=5000) # Start the Flask app
```
### model_interaction/prompting.py
```python
import sys
import together
together.api_key =""
def ask():
print("Say 'exit' in order to leave chat bot")
inp = ""
while True:
inp = input('AMA: ')
if inp == "exit":
sys.exit()
output = together.Complete.create(
prompt = f"{inp}\n:",
model = "lmsys/vicuna-13b-v1.5",
max_tokens = 256,
temperature = 0.8,
top_k = 60,
top_p = 0.6,
repetition_penalty = 1.1,
stop = ['', '\n\n']
)
# print generated text
print(output['prompt'][0]+output['output']['choices'][0]['text'])
def parserPrompt(x):
output = together.Complete.create(
prompt = f"{x}\n:",
model = "togethercomputer/llama-2-7b-chat",
max_tokens = 256,
temperature = 0.8,
top_k = 60,
top_p = 0.6,
repetition_penalty = 1.1,
stop = ['', '\n\n']
)
# print generated text
return output['output']['choices'][0]['text']
```
### contextual_inference/threaded_chunk_processing.py
```python
import threading
import together
import time
from requests.exceptions import HTTPError
together.api_key = ""
def inference(question: str, max_tokens=512) -> str:
prompt = f": {question}\n:"
output = together.Complete.create(
prompt=prompt,
model="lmsys/vicuna-13b-v1.5",
max_tokens=max_tokens,
temperature=0.7,
top_k=50,
top_p=0.7,
repetition_penalty=1,
stop=[":"],
)
return output["output"]["choices"][0]["text"]
def process_chunks_in_parallel(chunks, question):
relevant_info = []
# Define a function to process a single chunk
def process_chunk(chunk):
prompt = chunk
prompt += "\n**From the content above, summarize everything that may be relevant to the question: "
prompt += f'"{question}"**'
try:
result = inference(prompt, max_tokens=512)
relevant_info.append(result)
except HTTPError:
pass
# Create a list to hold thread objects
threads = []
# Start a thread for each chunk
for chunk in chunks:
thread = threading.Thread(target=process_chunk, args=(chunk,))
threads.append(thread)
thread.start()
# Wait for all threads to finish
for thread in threads:
thread.join()
return relevant_info
```
### contextual_inference/broski_inference.py
```python
import together
from math import ceil
from threaded_chunk_processing import process_chunks_in_parallel
MAX_CHUNK_SIZE = 20_000 # characters
together.api_key = ""
def inference(prompt: str, max_tokens=512) -> str:
output = together.Complete.create(
prompt=prompt,
model="lmsys/vicuna-13b-v1.5",
max_tokens=max_tokens,
temperature=0.7,
top_k=10,
top_p=0.7,
repetition_penalty=1,
stop=[":"],
)
return output["output"]["choices"][0]["text"].replace("*", "").replace('"', "").replace("'", "")
with open("compressed_posts.txt", "r", encoding="utf-8") as data_file:
compressed_data = data_file.read()
chunks = [
compressed_data[MAX_CHUNK_SIZE * i : MAX_CHUNK_SIZE * (i + 1)]
for i in range(ceil(len(compressed_data) / MAX_CHUNK_SIZE))
]
def ask_question(question: str) -> str:
relevant_info = process_chunks_in_parallel(chunks, question)
# Ask question
prompt = "\n".join(relevant_info)
prompt += "\n**Using the information above answer the following question "
"in a knolwedgable tone of voice. Refrain from using any vulgar or inappropriate "
"language. If conflicting views are present respond with the majority view, whilst "
"acknowledging the views of the larger minorities. Respond like a knowledgeable third party, do not mention data above, commentors, posters, etc.\n"
prompt += f'Question: "{question}"**'
return inference(prompt)
if __name__ == "__main__":
print(ask_question("Is CS 61A hard?"))
```
### model_interaction/parsing.py
```python
import json
import jsonlines
from datetime import datetime
import together
import prompting
import time
# Convert Unix timestamp to a readable date
def unix_to_readable_date(timestamp):
return datetime.utcfromtimestamp(int(timestamp)).strftime('%Y-%m-%d %H:%M:%S')
def processDataDefault():
input_file_path = "./raw_berkeley_posts.json"
output_file_path = "./outputraw.jsonl"
processData(input_file_path, output_file_path)
def processDataLLM(input_file, output_file):
with open(input_file, 'r') as infile:
data = json.load(infile)
c = 0
transformed_data = []
for item in data:
c += 1
print(c)
comments = item.get('comments', [])
if comments is None:
continue
questionData = [(item['title'], item['body'])]
answerData = []
for i in range(len(comments)):
if (i >= 3 or comments[i] is None):
break
answerData.append(comments[i].get('body', ''))
questionParaphrase = prompting.parserPrompt(f"Summarize this title and body into one question: {questionData[0][0]}\n{questionData[0][1]}")
time.sleep(0.1)
answerParaphrase = prompting.parserPrompt(f"Summarize these three comments and get the general idea: {' '.join(answerData)}")
time.sleep(0.1)
title_body = f": {questionParaphrase}"
# Assuming comments is a field in each item and is sorted in descending order of upvotes
formatted_item = {
"text": f": {answerParaphrase}"
}
transformed_data.append(formatted_item)
with open(output_file, 'w') as outfile:
for item in transformed_data:
json.dump(item, outfile)
outfile.write('\n') # Write each item on a new line
def processData(input_file, output_file):
with open(input_file, 'r') as infile:
data = json.load(infile)
transformed_data = []
c = 0
for item in data:
c+=1
print(c)
if "[deleted" in item['title'] or "[deleted" in item['body']:
continue
title_body = f": {item['title']} {item['body']}"
# Assuming comments is a field in each item and is sorted in descending order of upvotes
comments = item.get('comments', [])
if not comments or "[deleted" in comments[0].get('body', '') or "This post has been removed" in comments[0].get('body', ''):
continue
highest_upvoted_comment = comments[0].get('body', '')
formatted_item = {
"text": f"{title_body} : {highest_upvoted_comment}"
}
transformed_data.append(formatted_item)
with open(output_file, 'w') as outfile:
for item in transformed_data:
json.dump(item, outfile)
outfile.write('\n') # Write each item on a new line
def oldParser():
# Load the submissions
submissions_file = "./berkeley_submissions.jsonl"
submissions = {}
with open(submissions_file, 'r', encoding='utf-8') as f:
for line in f:
data = json.loads(line)
submissions[data['id']] = {
'title': data['title'],
'url': data['url'],
'body': data['selftext'],
'created_utc': data['created_utc'],
'date': unix_to_readable_date(data['created_utc'])
}
# Load and sort comments
comments_file = "./berkeley_comments.jsonl"
sorted_comments = []
with open(comments_file, 'r', encoding='utf-8') as f:
for line in f:
data = json.loads(line)
if data['body'] != '[deleted]' and data['link_id'][3:] in submissions:
submission_data = submissions[data['link_id'][3:]]
sorted_comments.append({
'submission_title': submission_data['title'],
'submission_url': submission_data['url'],
'submission_body': submission_data['body'],
'comment_body': data['body'],
'comment_score': data['score'],
'comment_created_utc': data['created_utc'],
'submission_created_utc': submission_data['created_utc'],
'submission_date': submission_data['date']
})
# Sort the comments by submission timestamp first, then comment timestamp
sorted_comments = sorted(sorted_comments, key=lambda x: (int(x['submission_created_utc']), int(x['comment_created_utc'])))
# Write to the file
output_file = "./parsed_data.txt"
with open(output_file, 'w', encoding='utf-8') as out:
for entry in sorted_comments:
out.write(f"Date: {entry['submission_date']}\nSubmission Title: {entry['submission_title']}\nSubmission URL: {entry['submission_url']}\nSubmission Body: {entry['submission_body']}\nComment: {entry['comment_body']}\nComment Score: {entry['comment_score']}\n\n")
# if __name__ == "__main__":
# processDataDefault()
```
### model_interaction/file_scraper/submission_extractor.py
```python
import json
file_path = r"D:\data\reddit\submissions\RS_2023-08"
assert True
berkeley_reddit_posts = []
with open(file_path, "r") as json_file:
for line in json_file:
if '"subreddit": "berkeley",' in line:
berkeley_reddit_posts.append(line)
with open("berkeley_posts_2023-08.json", "w") as json_file:
for line in berkeley_reddit_posts:
json_file.write(line)
```
### model_interaction/file_scraper/file_export.py
```python
import json
IN_PATH = "berkeley_posts_2023-09(1).json"
OUT_PATH = "berkeley_posts_2023-09(final).json"
all_data = []
with open(IN_PATH, 'r') as json_file:
for line in json_file:
all_data.append(json.loads(line))
# Load existing data from the file (if it exists)
try:
with open(OUT_PATH, 'r') as json_file:
existing_data = json.load(json_file)
except FileNotFoundError:
existing_data = []
# Append new data to the existing data list
existing_data.extend(all_data)
# Write the updated data (existing + new) back to the file
with open(OUT_PATH, 'w') as json_file:
json.dump(existing_data, json_file)
```
### model_interaction/file_scraper/api_posts_from_comments.py
```python
import praw
import json
from collections import defaultdict
# Initialize a Reddit API instance
reddit = praw.Reddit(
client_id='',
client_secret='',
user_agent='',
)
# Define your list of Reddit comments (each comment is a dictionary)
comments = []
with open("berkeley_posts_2023-08.json", "r") as json_file:
for line in json_file:
comments.append(json.loads(line))
# Categorize by link_id (posts)
comment_link_ids = defaultdict(list)
for comment in comments:
comment = {
"body": comment["body"],
"id": comment["id"],
"link_id": comment["link_id"],
"upvote_count": comment["score"]
}
comment_link_ids[comment["link_id"]].append(comment)
# Function to retrieve post information
def get_post_info(link_id):
post_id = link_id.split('_')[1]
submission = reddit.submission(id=post_id)
return {
"title": submission.title,
"body": submission.selftext,
"upvote_count": submission.score,
}
# Iterate through the list of comments and fetch post information
posts = []
for link_id, post_comments in comment_link_ids.items():
post_info = get_post_info(link_id)
# Skip if photo post
if not post_info["body"]:
continue
post_info["comments"] = sorted(post_comments, key=lambda c: -c["upvote_count"])[:5]
posts.append(post_info)
# Save data to json file
try:
with open("berkeley_posts_2023-08(1).json", 'w') as json_file:
for post in posts:
json.dump(post, json_file)
json_file.write('\n')
except Exception as e:
print(f"An error occurred while writing to the JSON file: {e}")
```
[2 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]