# Project export: unbias This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Accessibility focused browser extension that helps you communicate more effectively with the use of multiples types of AI. - Devpost: https://devpost.com/software/unbias-ytae79 - GitHub: https://github.com/isaac-h-chan/unbias - Team: 2 GitHub contributor(s) — Jonathan Nguyen (26 commits), isaac-h-chan (20 commits) ## Devpost submission (written by the team) ### Inspiration We noticed the lack of AI integration in web browsers and took the opportunity to fill that vacuum with an innovative project. One of the primary ways in which people interact with web browsers, is by writing content. We leveraged the power of AI help people communicate on the web. ### What it does The browser extension leverages three different types of AI to improve communication on the web. The extension has a sophisticated classification model to help users determine whether they are writing in sufficient complexity (or simplicity) for their intended audience. The extension also performs sentiment analysis to help users identify whether (and how much) their speech effectively communicates what they are saying in a respectful and positive manner. Finally, the extension makes use of the power of generative AI to help users better express their perhaps non-agreeable opinions in a friendlier way. ### How we built it We created the frontend of the extension using javascript, html, and css. Our backend and API was built using FastAPI and together.ai's Together API ### Challenges we ran into There are no external frameworks for developing web extension technologies, so it was an inconvenience to have to develop everything using vanilla javascript, html and css. ### Accomplishments we're proud of We are very proud of the fact that we were able to leverage so many different types of AI, effectively, in such a short amount of time. ### What we learned We learned how to build an API using FastAPI and we also learned about the intricacies and practices of developing extensions for web browsers such as Chrome. ### What's next We want to improve our extension by adding new features such as providing a variety of options in which the extension can rephrase text in different tones. ## README (from the GitHub repository) # Unbias Chrome Extension A browser extension to help you better express yourself **unbias** is a revolutionary Chrome extension designed to help users with a variety of accessibility issues. Our mission is to ensure clear, concise, and neutral communication in digital interactions. Whether you're concerned about accidentally offending someone due to unintended tone, or want to ensure your message isn't misunderstood, **unbias** is here to assist you. **unbias** was made at CalHacks 10.0 (2023), check out our devpost [here](https://devpost.com/software/unbias-ytae79) ### Features - **Reading Level Assessment**: Get a grade-level estimate for any highlighted text - **Tone Analysis**: Sentences are individually evaluated for tone, with a compound score representing the average of the positive, negative, and neutral tones - **AI-Powered Sentence Recommendations**: Click on a sentence in the popup to fetch an AI-recommended replacement, structured to be more neutral and clear ### How to Use 1. **Highlight Text**: Highlight a portion of text on any web page. 2. **Activate unbias**: Click on the unbias icon in the top right of your Chrome browser. 3. **View Insights**: View the reading level grade in the popup. 4. **Analyze Tone**: Sentences from the highlighted text are graded based on tone. 5. **Get Recommendations**: Click on any sentence in the popup to receive a recommended replacement. ### License This project is licensed under the MIT License. See the LICENSE file for details. ## Detected evidence (automated analysis) Indexed codebase: 9 recognized source files, 16 KB. - CSS (language) — detected in the code - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code ## Codebase structure (from repository index) ### Files (13 of 13) ``` .gitignore api/analyze.py api/api.py api/request.py front-end/background.js front-end/popup/popup.css front-end/popup/popup.html front-end/popup/popup.js front-end/scripts/content.js LICENSE manifest.json README.md requirements.txt ``` ### Dependencies - requirements.txt: annotated-types@==0.6.0, anyio@==3.7.1, certifi@==2023.7.22, charset-normalizer@==3.3.1, click@==8.1.7, exceptiongroup@==1.1.3, fastapi@==0.104.0, h11@==0.14.0, httptools@==0.6.1, idna@==3.4, joblib@==1.3.2, nltk@==3.8.1, pydantic@==2.4.2, pydantic_core@==2.10.1, python-dotenv@==1.0.0, PyYAML@==6.0.1, regex@==2023.10.3, requests@==2.31.0, sniffio@==1.3.0, sseclient-py@==1.8.0, starlette@==0.27.0, tabulate@==0.9.0, together@==0.2.6, tqdm@==4.66.1, typer@==0.9.0, typing_extensions@==4.8.0, urllib3@==2.0.7, uvicorn@==0.23.2, uvloop@==0.19.0, watchfiles@==0.21.0, websockets@==12.0 ### Recent commits (newest first) - Update README.md - Update README.md - log click in console - changed png - Update README.md - Update README.md - Update README - minor ui changes - Update manifest.json permissions - Update api.py - Merge pull request #10 from isaac-h-chan/feature - implemented sentence reccomendations - added css and html injecting - Merge pull request #9 from isaac-h-chan/feature - calculated clarity index - fixed complexity score - made improvements to analysis, text rephrasing and added js function to call neutral endpoint - output data to popup - fixed divide by zero edge case - Use async to fulfill promise error ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` annotated-types==0.6.0 anyio==3.7.1 certifi==2023.7.22 charset-normalizer==3.3.1 click==8.1.7 exceptiongroup==1.1.3 fastapi==0.104.0 h11==0.14.0 httptools==0.6.1 idna==3.4 joblib==1.3.2 nltk==3.8.1 pydantic==2.4.2 pydantic_core==2.10.1 python-dotenv==1.0.0 PyYAML==6.0.1 regex==2023.10.3 requests==2.31.0 sniffio==1.3.0 sseclient-py==1.8.0 starlette==0.27.0 tabulate==0.9.0 together==0.2.6 tqdm==4.66.1 typer==0.9.0 typing_extensions==4.8.0 urllib3==2.0.7 uvicorn==0.23.2 uvloop==0.19.0 watchfiles==0.21.0 websockets==12.0 ``` ### api/request.py ```python import requests import analyze import json '''response = requests.post("http://127.0.0.1:8000/analyze", json={ "text": "If you don't get your act together, I'm going to lose my mind!!!I'm not kidding!!!I'm at my wit's end!!!I've never been so angry in my life!!!" } ) print(response.content) response2 = requests.post("http://127.0.0.1:8000/analyze", json={"text": "We turn DMs into more flexible conditional image generators by augmenting their underlying UNet backbone withthe cross-attention mechanism [97], which is effective for learning attention-based models of various input modalities [35,36]."}) print(response2.content)''' """res3 = requests.post("http://127.0.0.1:8000/neutral", json={ "text": "I'm so angry right now that I could spit nails. You've had every opportunity to succeed, but you've thrown it all away. You're a disappointment to me and to everyone else who believes in you." } ) print(res3.content.decode('utf-8'))""" res4 = requests.post("http://127.0.0.1:8000/analyze", json={"text": "You are degenerate, noxious and depraved."}) print(res4.content) ``` ### front-end/background.js ```javascript chrome.webNavigation.onCompleted.addListener((details) => { chrome.scripting.executeScript({ target: {tabId: details.tabId}, files: ['front-end/scripts/content.js'] }); }); chrome.tabs.onActivated.addListener((activeInfo) => { chrome.scripting.executeScript({ target: {tabId: activeInfo.tabId}, files: ['front-end/scripts/content.js'] }); }); let popupSender = null; // makes api call chrome.runtime.onMessage.addListener(async (message, sender) => { if (message.action === 'makeApiCall') { const selectedText = message.text; popupSender = sender; try { let data = await getScores(selectedText); chrome.runtime.sendMessage(popupSender.id, { action: 'apiResponse', data: data }); // log data for testing // console.log('text complexity: ' + data.complexity); // console.log('text length: ' + data.length); // for (let i = 0; i < data.sents.length; i++) { // console.log('Sentence:', data.sents[i]); // console.log('Sentiment Score:', data.sentiment[i].compound); // } // console.log(data); } catch (error) { console.error("Error processing data: ", error); } } }); async function getScores(text) { const url = "http://127.0.0.1:8000/analyze"; const request = new Request(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text: text }) // Use JSON.stringify here }); try { const response = await fetch(request); const data = await response.json(); return data; } catch (error) { console.error("Error making API request:", error); } } ``` ### api/api.py ```python from fastapi import FastAPI from pydantic import BaseModel from api.analyze import getMeanSyllables, getMeanWords, getSentiment, cleanTokenize import regex as re import together together.api_key = "YOUR-KEY-HERE" ENDPOINT_URL = "https://api.together.xyz/inference" MODEL = "togethercomputer/llama-2-70b-chat" class Words(BaseModel): text: str app = FastAPI() @app.post("/analyze") async def analyze(req: Words): text = req.text mWords = getMeanWords(text) mSyllables = getMeanSyllables(text) sents, sentiment = getSentiment(text) return { "sents": sents, "length": mWords, "complexity": mSyllables, "sentiment": sentiment } @app.post("/neutral") def neutral(req: Words): text = req.text maxTokens = 2*(len(cleanTokenize(text)).bit_length() + 1)**2 prompt_stem = """: Rephrase the following text in a neutral tone: If you don't get your act together, I'm going to lose my mind!!!I'm not kidding!!!I'm at my wit's end!!!I've never been so angry in my life! : I'm feeling overwhelmed and concerned about the situation. I would appreciate it if you could help me find a solution. : Rephrase the following text in a neutral tone: You had one simple job and you couldn't even do that right. I'm so sick and tired of your incompetence. : I'm disappointed that you didn't complete the task as expected. I'm concerned about your performance and I'd like to discuss how we can improve it. : Rephrase the following text in a neutral tone:""" res = together.Complete.create( model=MODEL, prompt=prompt_stem + text + "\n\n:", max_tokens=maxTokens, stop=["", "\n\n"], temperature=0.7, top_p=0.7, top_k=50, repetition_penalty=1 ) return { "text": re.sub("\\n\\n", "", res["output"]["choices"][0]["text"]) } ``` ### api/analyze.py ```python import regex as re import nltk import ssl import json from nltk.tokenize import SyllableTokenizer from nltk.sentiment import vader from nltk.corpus import stopwords try: _create_unverified_https_context = ssl._create_unverified_context except AttributeError: pass else: ssl._create_default_https_context = _create_unverified_https_context nltk.download('vader_lexicon') nltk.download('stopwords') nltk.download('punkt') ssp = SyllableTokenizer() sentimentAnalyzer = vader.SentimentIntensityAnalyzer() # Processes a single string def cleanTokenize(words): if not isinstance(words, str): words = str(words) words = words.lower() # remove emojis emoji_pattern=re.compile(pattern = "[" u"\U0001F600-\U0001F64F" # emoticons u"\U0001F300-\U0001F5FF" # symbols & pictographs u"\U0001F680-\U0001F6FF" # transport & map symbols u"\U0001F1E0-\U0001F1FF" # flags (iOS) "]+", flags = re.UNICODE) words = emoji_pattern.sub(r'',words) # remove digits and punctuation words = words.translate(words.maketrans('!"#%&$()*+-./:;,<=>?@[\\]^_`{|}~0123456789', ' '*41)) words = re.sub('\s+',' ', words).strip() return nltk.tokenize.word_tokenize(words) def getMeanSyllables(text: str) -> float: """ Args: - text: a raw, unprocess block of text Returns: Float value representing average syllables in the block of text excluding stopwords """ words = cleanTokenize(text) meanSyllables = 0 for word in words: tokens = ssp.tokenize(word) meanSyllables += len(tokens) meanSyllables /= max(1, len(words)) return meanSyllables def getSentences(text: str) -> list: return nltk.sent_tokenize(text) def getMeanWords(text: str) -> float: """ Args: - text: a raw, unprocess block of text Returns: - Float value representing the average number of words per sentence in the text """ meanWords = 0 sents = [re.sub("[.!?]", "", sent) for sent in getSentences(text)] meanWords += sum([len(sent.split()) for sent in sents]) return meanWords/max(1, len(sents)) def getSentiment(text: str) -> list and list: """ Args: - text: a raw, unprocess block of text Returns: - List containing the unique sentences in the text - List containing the polarity scores of the sentences in the corresponding index """ sentiment = [] sents = getSentences(text) for sent in sents: sentiment.append(sentimentAnalyzer.polarity_scores(sent)) return sents, sentiment ``` ### front-end/scripts/content.js ```javascript function getSelectedText() { const selection = window.getSelection(); return selection.toString(); } chrome.runtime.onMessage.addListener((request, sender, sendResponse) => { if (request.action === 'getSelectedText') { const selectedText = getSelectedText(); if (selectedText) { sendResponse({ action: 'selectedText', text: selectedText }); } else { sendResponse({ action: 'selectedText', text: null }); } } }); ``` ### front-end/popup/popup.html ```html Unbias ``` ### front-end/popup/popup.css ```css body { font-family: 'Arial', sans-serif; background-color: #404040; margin: 0; padding: 8px; width: 320px; } #container { background-color: #ffffff; box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1); padding: 15px; overflow: hidden; border-radius: 10px; } h1 { font-size: 48px; color: #333; margin-top: 0; margin-bottom: 10px; text-align: center; font-weight: bold; background: -webkit-linear-gradient( rgba(255, 0, 0, 1), rgba(255, 0, 180, 1), rgba(0, 100, 200, 1)); -webkit-background-clip: text; -webkit-text-fill-color: transparent; } #selectedText { display: flex; color: #555; margin-bottom: 10px; font-size: 18px; justify-content: center; } hr { border-top: 1px solid #eee; margin: 10px 0; } #complexity { display: flex; justify-content: center; color: #555; font-size: 18px; } ul { /* text-align: justify; */ margin-top: 0px; padding-left: 12px; padding-right: 12px; font-size: 18px; font-weight: bold; } #sentences > li { font-weight: normal; list-style-type: none; margin: 0; padding: 0; } li:hover, .sent:hover { background-size: 0 3px, 100% 3px; color: black; } li { background: linear-gradient( to right, rgba(100, 200, 200, 1), rgba(100, 200, 200, 1) ), linear-gradient( to right, rgba(255, 0, 0, 1), rgba(255, 0, 180, 1), rgba(0, 100, 200, 1) ); background-size: 100% 3px, 0 3px; background-position: 100% 100%, 0 100%; background-repeat: no-repeat; transition: background-size 400ms; } #sentences li { padding: 5px 0; color: #666; display: flex; align-items: center; } .sent { width: 85%; padding-right: 8px; } .score { width: 15%; padding: 2px; text-align: right; font-weight: bolder; } ``` ### front-end/popup/popup.js ```javascript // receives data from background.js if api call is made chrome.runtime.onMessage.addListener((message, sender) => { if (message.action !== 'apiResponse') { return; } const data = message.data; const sentences = data.sents; const scores = data.sentiment; const grade = grades[Math.min(Math.floor((0.39 * data.length) + 11.8 * data.complexity - 15.59), 18)]; document.getElementById('complexity').textContent = 'Comprehension Level: ' + grade; document.getElementById('body').style.width = '400px'; const container = document.getElementById('container'); // hides comprehension level and line breaks when no text selected const hiddenContent = document.getElementById('hidden'); hiddenContent.removeAttribute('hidden'); // injects tone scores list const sent_list = document.createElement('ul'); sent_list.innerText = 'Tone Scores'; sent_list.setAttribute('id', 'sentences'); container.appendChild(sent_list); // injects sentence into tone scores list const sentsList = document.getElementById('sentences'); for (i = 0; i < sentences.length; ++i) { let li = document.createElement('li'); li.classList.add('element') let sent = document.createElement('div'); sent.classList.add('sent') sent.innerText = sentences[i] let score = document.createElement('div'); score.classList.add('score') score.innerText = scores[i].compound.toFixed(2); li.appendChild(sent); li.appendChild(score); sentsList.appendChild(li); const text = sent.innerText; li.addEventListener('click', () => { const text = sent.innerText; // Adjusted this line to get the innerText within the event listener's scope const sentenceRequest = new Request("http://127.0.0.1:8000/neutral", { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text: text }) }); // generate new sentence let newSentence; generate(sentenceRequest) .then(newSent => { newSentence = newSent.text; sent.innerText = newSentence; // Now use the newSent to make the second request return generate(new Request("http://127.0.0.1:8000/analyze", { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text: newSentence }) })); }) .then(newScore => { score.innerText = newScore.sentiment[0].compound; }) .catch(error => { console.error("Error in generate function:", error); }); console.log('sentence clicked!'); }); } }) chrome.tabs.query({ active: true, currentWindow: true }, (tabs) => { const currentTab = tabs[0]; if (currentTab.url?.startsWith("chrome://")) { document.getElementById('selectedText').textContent = "Does not work in chrome:// websites"; return undefined; } // Request the selected text from the content script chrome.tabs.sendMessage(currentTab.id, { action: 'getSelectedText' }, (response) => { if (response && response.text) { document.getElementById('selectedText').textContent = "Scores of the Selected Text"; // Notify background script to make the API call chrome.runtime.sendMessage({ action: 'makeApiCall', text: response.text }); } else { document.getElementById('selectedText').textContent = 'Select text to begin!'; } }); }); grades = { 0: 'Kindergarten', 1: 'First Grade', 2: 'Second Grade', 3: 'Third Grade', 4: 'Fourth Grade', 5: 'Fifth Grade', 6: '6th to 7th Grade', 7: '6th to 7th Grade', 8: '8th to 9th Grade', 9: '8th to 9th Grade', 10: '10th to 12th Grade', 11: '10th to 12th Grade', 12: '10th to 12th Grade', 13: 'College Level', 14: 'College Level', 15: 'College Level', 16: 'College Level', 17: 'Graduate Level', 18: 'Graduate Level' } const generate = (request) => { return fetch(request) .then((response) => { if (!response.ok) { throw new Error('Response error'); } return response.json(); }) .catch((error) => { console.error(error); throw error; }); } ```