# Project export: Recap

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 10.0
- Tagline: Discover your ideal classes and professors effortlessly. Our Reddit-powered search engine offers insightful results to streamline your academic journey.
- Devpost: https://devpost.com/software/classfinder-r4aw02
- GitHub: https://github.com/Lojikick/Recap
- Video: https://www.youtube.com/embed/8uJDhbn93Y4?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Lojikick (8 commits), Shashwat Bansal (6 commits)

## Devpost submission (written by the team)

### Inspiration

As enrollment season approached, one of our team members found themselves endlessly scrolling through Reddit, desperately seeking insights into specific Math 104 professors. Their exhaustive search left them empty-handed, prompting us to consider how valuable the wealth of knowledge and firsthand experiences shared by students on Reddit could be for the wider academic community. With the recent surge in Reddit's popularity, it only seemed fitting to harness this collective wisdom for the greater good.

### What it does

Recap is a platform designed to assist students in finding relevant information about courses and professors. Users input the name of a course and professor, and Recap provides a curated selection of comments and posts related to that class. It also offers insights such as sentiment analysis for each comment and an assessment of the overall difficulty of the class.

### How we built it

Our journey began with the integration of the Reddit API, known as PRAW. We set up a developer account on Reddit to gain access to their database and enable us to perform queries effectively. PRAW's innate ability to sort posts and comments by relevance was a substantial time-saver, allowing us to focus on extracting insights. We devised a preliminary algorithm for measuring course difficulty, which can undoubtedly be refined in the future. To determine sentiment, we employed Vader Sentiment for sentiment analysis, and for the sake of clarity and accessibility, we transformed sentiment scores into emojis. While some of the metrics were simplified due to time constraints, they served our purpose well.

### Challenges we ran into

Initially, we explored the possibility of using Convex for our project, but we soon realized that we were faced with a learning curve that exceeded our technical expertise, particularly concerning Backend TypeScript. Reflex appeared to be a promising alternative, but server connection issues hindered our progress. Two-thirds of our team underestimated the complexity of building a web app, especially the Frontend development aspect.

### Accomplishments we're proud of

We take pride in bridging the Reddit community with students seeking invaluable information that was previously shared only through word of mouth. Our sentiment analysis and course difficulty assessments equip students with realistic expectations, enabling them to make informed academic choices confidently.

### What we learned

This journey revealed that full-stack development is significantly more challenging than it might appear at first. Despite our limited knowledge of Frontend technologies like CSS and HTML, we managed to create a functional web app by using Flask and adding features like emojis in our final table view. We also learned about the rapid pace of real-world technology. In terms of data analysis, we made intriguing discoveries about the Berkeley subreddit (and about how much harder our classes are to Stanfords') and the vocabulary used to describe course experiences. We also encountered controversial insights about the popularity of certain professors compared to others.

### What's next

Recap is poised to become an essential tool for every student. We envision its integration with websites like Berkeleytime to expand its reach. As we continue to refine our product, we aim to enhance its mathematical underpinnings, comprehend the limitations and contexts of our results, and improve accuracy through rigorous testing. There are glaring issues with data cleaning and accessibility that need to be fixed. While assessing course difficulty remains subjective, we are committed to refining our approach and striving for greater precision.

## README (from the GitHub repository)

https://devpost.com/software/classfinder-r4aw02

## Detected evidence (automated analysis)

Indexed codebase: 6 recognized source files, 12 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- Python (language) — detected in the code

## Codebase structure (from repository index)

### Files (9 of 9)

```
app.py
backEnd.ipynb
LICENSE
README.md
requirements.txt
static/style.css
templates/account.html
templates/home.html
templates/template.html
```

### Dependencies

- requirements.txt: blinker@==1.7.0, certifi@==2023.11.17, charset-normalizer@==3.3.2, click@==8.1.7, colorama@==0.4.6, Flask@==3.0.0, idna@==3.6, itsdangerous@==2.1.2, Jinja2@==3.1.3, joblib@==1.3.2, MarkupSafe@==2.1.3, nltk@==3.8.1, numpy@==1.26.3, pandas@==2.1.4, praw@==7.7.1, prawcore@==2.4.0, python-dateutil@==2.8.2, pytz@==2023.3.post1, regex@==2023.12.25, requests@==2.31.0, six@==1.16.0, tqdm@==4.66.1, tzdata@==2023.4, update-checker@==0.18.0, urllib3@==2.1.0, websocket-client@==1.7.0, Werkzeug@==3.0.1, wheel@==0.42.0

### Recent commits (newest first)

- Added requirements.txt
- Change README.txt
- hyperlink
- white text
- rounding fix
- fixed nonetype bug
- Added displays for difficulty and overall sentiment
- saved avg sentiment and difficulty
- Merge branch 'main' of https://github.com/Lojikick/Recap
- Massive Display Changes
- hardness
- minor cleaning
- ?
- added comment support to app.py
- ?
- emoji
- titles
- Stylized Dataframe!
- M
- Avik: Got site to output simple pandas dataframe

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
blinker==1.7.0
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
Flask==3.0.0
idna==3.6
itsdangerous==2.1.2
Jinja2==3.1.3
joblib==1.3.2
MarkupSafe==2.1.3
nltk==3.8.1
numpy==1.26.3
pandas==2.1.4
praw==7.7.1
prawcore==2.4.0
python-dateutil==2.8.2
pytz==2023.3.post1
regex==2023.12.25
requests==2.31.0
six==1.16.0
tqdm==4.66.1
tzdata==2023.4
update-checker==0.18.0
urllib3==2.1.0
websocket-client==1.7.0
Werkzeug==3.0.1
wheel==0.42.0

```

### app.py

```python
from flask import * #importing flask (Install it using python -m pip install flask)

from flask import *
import praw
import numpy as np
import pandas as pd
import nltk
nltk.download('vader_lexicon')
from nltk.sentiment.vader import SentimentIntensityAnalyzer
pd.options.mode.chained_assignment = None


app = Flask(__name__) #initialising flask

# Reddit API credentials
client_id = 'Lnt2q1aiGONcbox9EyK-Mw'
client_secret = 'VdTP86mTMJIPop59Xeks_Ldr_AgWCQ'
user_agent = 'test'  # A description of your application

# Initialize PRAW
reddit = praw.Reddit(client_id=client_id, client_secret=client_secret, user_agent=user_agent)

@app.route("/") #defining the routes for the home() funtion (Multiple routes can be used as seen here)
@app.route("/home")
def home():
    return render_template("home.html") #rendering our home.html contained within /templates

@app.route("/account", methods=["POST", "GET"]) #defining the routes for the account() funtion
def account():
    res = "<User Not Defined>" #Creating a variable usr

    if (request.method == "POST"): #Checking if the method of request was post
        res = request.form["prompt"] 
        school = request.form["school"] 
        data = score_keyword(res, school, 75)
        avg_sentiment = round(calculate_average_sentiment(data), 2)
        avg_difficulty = round(calculate_average_difficulty(data)*10, 1)
        avg_difficulty = str(avg_difficulty) + "/10.0"
        if (data.shape[0] > 0):
            data['Sentiment'] = data['Sentiment'].apply(add_emoji)
        # data = { 'Column1': [10, 20, 30], 'Column2': ['A', 'B', 'C'] }
        df = pd.DataFrame(data)

        if not res: #if name is not defined it is set to default string
            res = "<User Not Defined>"
    # return render_template("account.html",results=res, data = output.to_html()) #rendering
    temp = "Hello"
    div_elements = ''
    rows = list(df.index)
    for row in range(len(rows)):
        author = df.iloc[row,3]
        comment = df.iloc[row,1][0:100]
        link = df.iloc[row,2]
        sentiment = df.iloc[row,4] 

        div_elements += f'<div>{author}&nbsp;&nbsp;&nbsp;<a href={link} target="_blank">{comment}</a>&nbsp;&nbsp;&nbsp;{sentiment}</div>'
    return render_template("account.html", results=res, divs=div_elements, sent = avg_sentiment, diff = avg_difficulty) 

    # our account.html contained within /templates
    # display_data(res, df)


#Major Backend Functions----
# def display_data(res, df):
#     div_elements = ''
#     for column in df.columns:
#         for value in df[column]:
#             div_elements += f'<div>{value}</div>'
    
   

def search_reddit(search_word, subreddit_name, search_limit):
    # Search for comments related to the specified word
    if subreddit_name:
        subreddit = reddit.subreddit(subreddit_name)
        submissions = subreddit.search(search_word, limit=search_limit)  # You can adjust the limit as needed
    else:
        submissions = reddit.subreddit('all').search(search_word, limit=search_limit)
    authors, comments, urls, titles = [], [], [], []
    # Iterate through the comments and extract their data
    counter = 0
    for submission in submissions:
        if (counter >= search_limit):
            break
        slftxt = submission.selftext
        url = submission.url
        if (slftxt.find('Piazza') != -1 or url[-4:] == '.jpg'):
            continue
        authors.append(submission.author.name)
        comments.append(slftxt)
        urls.append(url)
        titles.append(submission.title)
        counter+=1
        for comment in submission.comments:
            if (counter >= search_limit):
                break
            if (comment.author):
                authors.append(comment.author.name)
            else:
                authors.append('')
            comments.append(comment.body)
            urls.append(submission.url)
            titles.append(submission.title)
            counter+=1
    raw_data = pd.DataFrame({'Title': titles, 'Comment':comments, 'URL':urls, 'Author':authors})
    return raw_data
analyzer = SentimentIntensityAnalyzer()
def get_emotion_emoji(score):
        if score < -0.6:
            return "😢"  # Very sad
        elif score < -0.2:
            return "😔"  # Somewhat sad
        elif score > 0.6:
            return "😄"  # Very happy
        elif score > 0.2:
            return "😊"  # Somewhat happy
        else:
            return "😐"  # Neutral
def process_data(data):
    new_data = data.copy()
    comments = new_data['Comment']
    comments = comments.str.lower()
    comments = comments.str.replace(r'http.*($|\s)', '') # Get rid of images
    comments = comments.str.replace(r'[^\w|_|\s]|\n', ' ') # Get rid of punctuation and newlines
    new_data['Split'] = comments.str.split() # Create split column
    new_data = new_data[new_data['Split'].str.len() > 1] # Remove empty or one word rows
    new_data = new_data.set_index('Author')
hard_words = ['fail', 'failing', 'suck', 'hard', 'difficult', 'dumb', 'drop', 'dropping', 'droppin', 'desperate', 'impossible',
    "hard", "strenuous", "arduous", "laborious", "heavy", "tough",
    "onerous", "burdensome", "demanding", "punishing", "grueling",
    "grinding", "back-breaking", "painful", "exhausting", "tiring",
    "fatiguing", "wearing", "wearying", "wearisome", "hellish",
    "killing", "knackering", "toilsome", "exigent", "problematic",
    "puzzling", "baffling", "perplexing", "confusing", "mystifying",
    "mysterious", "complicated", "complex", "involved", "intricate",
    "knotty", "thorny", "ticklish", "obscure", "abstract", "abstruse",
    "recondite", "enigmatic", "impenetrable", "unfathomable", "over one's head",
    "above one's head", "beyond one", "fiddly", "sticky",
    "gnarly", "wildering", "involute", "involuted", "😭", ":(", "wtf", "struggling", "fuck", "bad", "hurt", "dogshit", "challenging", "challenge"]
easy_words = ["easy", "simple", "effortless", "light", "gentle", "smooth",
    "easygoing", "simple",
[truncated — 3085 more characters]
```

### templates/account.html

```html
{% extends "template.html" %} {% block body %}

<br>
<h3>Class: {{results}}</h3>
<h3>Overall Sentiment: {{sent}}</h3>
<h3>Overall Difficulty: {{diff}}</h3>

<h3>Results for {{results}}</h3>
<!-- <h3>{{data | safe}}</h3> -->
<div class="data-divs">
    {{ divs | safe }}
</div>

<br>

{% endblock %}
```

### templates/home.html

```html
{% extends "template.html" %} {% block body %}

<h2>Home Page</h2>
<p>Type in your class and school</p>

<form action="/account" method="POST">
    <input type="text" name="prompt" id="prompt" placeholder="Type Class" class="form-feild"><br>
    <input type="text" name="school" id="school" placeholder="Type School" class="form-feild"><br>
    <button type="submit" class="add-btn">See Results</button>
</form>
<br>

{% endblock %}
```

### templates/template.html

```html
<html>
<!-- This is the template file. Any code written here will automatically be carried onto the files that extend from this file(eg: home.html, account.html) -->
<head>
    <link rel="stylesheet" href="../static/style.css">
    <title>Recap</title>
    <style>
        body {
            font-family: Arial, sans-serif; /* Changed the font to a more common sans-serif font */
            background-color: #3c3c3c; /* Light gray background */
            margin: 0;
            padding: 20px;
            color: #e8e8e8; /* Darkened the default text color */
        }

        h1 {
            text-align: center;
            color: #0aff0a; /* Retained the color for the h1 heading */
        }

        h3 {
            text-align: center;
            color: #e8e8e8; /* Retained the color for the h1 heading */
        }

        center {
            margin-top: 20px; /* Adjusted margin for center elements */
        }
    </style>
</head>

<body>
    <div>
        <h1>Recap</h1>
    </div>
    <div>
        <center>
            {% block body %} {% endblock %}
        </center>
    </div>
</body>

</html>
```

### static/style.css

```css
h1,
h2,
h3,
h4,
p {
    font-family: Arial, sans-serif;
    color: #FFF; /* Set text color to black */
}

body {
    font-family: Arial, sans-serif;
    background-color: #f5f5f5; /* Light gray background color */
    margin: 0;
    padding: 20px;
    color: #666; /* Adjust default text color */
}

:root {
    color: #d5e0e7; /* Retained the color variable */
}

.add-btn {
    height: 40px;
    width: 150px;
    color: #FFFFFF;
    background-color: #03d603;
    border: 1px solid #029902;
    border-radius: 5px;
    padding: 10px;
    margin: 10px;
}

.add-btn:hover,
.add-btn:focus {
    color: #029902;
    background-color: #00ff002f;
    border: 1px solid #029902;
    border-radius: 5px;
}

.form-field {
    height: 40px;
    width: 200px;
    border: 1px solid #99AAB5;
    border-radius: 5px;
    padding: 10px;
    margin: 10px;
    background-color: #f5f5f5; /* Set background color to match the body */
}

/* .data-divs {
    display: flex;
    flex-wrap: wrap;
} */
.data-divs div {
    border: 1px solid  #f5f5f5;
    background-color: #f5f5f5;
    color: #000;
    padding: 5px;
    margin: 5px;
}

.data-words div {
    border: 1px solid #d800cd;
    padding: 5px;
    margin: 5px;
}
```