# Project export: Miss Info

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2024
- Tagline: Detect online misinformation.
- Devpost: https://devpost.com/software/mrs-info
- GitHub: https://github.com/inkyant/cruz-hacks-2024
- Result: winner (Wildest Idea)
- Team: 3 GitHub contributor(s) — inkyant (35 commits), Jian Li (16 commits), Brennan0 (6 commits)

## Devpost submission (written by the team)

### Inspiration

In the age of rapid communication, social media platforms such as TikTok, Instagram, and YouTube have risen to the top in short-form media. These clips can circulate the internet and go viral. However, many of these viral videos often lead to the spreading of misinformation. How can someone quickly and easily check if a video they see is misleading?

### What it does

Detects misinformation in TikTok posts with AI. It provides a summary on potential misinformation present and links to any relevant articles about the topic.

### How we built it

The webpage was built in React while the backend utilized Flask and several AI libraries. These include OpenCV, LLama, ChatGPT, Assembly-AI, and a CDN is used to host video frames while processing. Additionally, web scraping of the fact-checking site snopes.com is done so that certain claims can be backed up with sources if they have been checked by snopes.com.

### Challenges we ran into

Obtaining the audio file from TikTok posts proved difficult, and the subsequent challenge involved aiming for high accuracy in detecting false information using the OpenAI API. However, by utilizing several APIs we were able to provide more context to improve the reliability. Additionally, scraping snopes.com for relevant articles provides sources for claims and prevents AI hallucination. We also had to learn how to use asynchronous programming to improve the speed of the program.

### Accomplishments we're proud of

We are proud of our ability to integrate several different APIs to tackle this difficult problem. The UI for the webpage was designed with care and user-friendly.

### What we learned

Our group learned to implement several APIs in our backend along with web scraping and image processing while collaborating with each other to hook up the frontend and backend.

### What's next

Develop a language model to aim for higher accuracy.

## README (from the GitHub repository)

# Miss Info (CruzHacks 2024)

A web page that detects any misinformation present in TikTok videos using AI and Large Language Models. The LLMs used for this project include LLama, ChatGPT, and Assembly-AI. OpenCV and CDN are also used to process video frames. After the provided TikTok video is processed, a summary of the misinformation analysis is displayed alongside relevant sources from [snopes.com](https://www.snopes.com). (Project for CruzHacks 2024)

![Screen Shot 2024-01-21 at 4 14 13 AM](https://github.com/inkyant/cruz-hacks-2024/assets/86862325/3f114ac2-ab5b-4b18-9644-d9bf31f7b6f5)

# Usage

Insert your OpenAI, AssemblyAI, Replicate, and CDN keys in [backend/api/api_keys.py](https://github.com/inkyant/cruz-hacks-2024/blob/main/backend/api/api_keys.py) with the following variables:
```
OPEN_AI_API_KEY = ""
ASM_AI_API_KEY = ""
REPLICATE_API_KEY = ""
CDN_CLOUD_NAME = ""
CDN_API_KEY = ""
CDN_API_SECRET = ""
```

This project consists of two components: the React application for webpage rendering and the Flask application for running the LLM APIs and processing the TikTok video. Both components must be run for the web application to function.

In one terminal, run the following commands to start up the React application:

```
cd client
npm install
npm run start
```

In another terminal, run the following commands to start up the Flask application:

```
pip3 install -r requirements.txt
cd backend
python3 app.py
```


## Detected evidence (automated analysis)

Indexed codebase: 17 recognized source files, 24 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Streamlit (technology) — detected in the code

## Codebase structure (from repository index)

### Files (20 of 20)

```
.gitignore
backend/api/__init__.py
backend/api/api_keys.py
backend/api/asmAI.py
backend/api/captions.py
backend/api/chatGPT.py
backend/api/image_cap.py
backend/api/routes.py
backend/api/snopes_check.py
backend/app.py
backend/tests/test_send.py
client/package.json
client/public/index.html
client/README.md
client/src/App.css
client/src/App.js
client/src/index.css
client/src/index.js
README.md
requirements.txt
```

### Dependencies

- client/package.json: @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, react@^18.2.0, react-dom@^18.2.0, react-loader-spinner@^6.1.6, react-scripts@5.0.1, react-social-media-embed@^2.4.1, web-vitals@^2.1.4
- requirements.txt: assemblyai, browser-cookie3, bs4, cloudinary, Flask, Flask-Cors, lxml, numpy, openai, opencv-python, pandas, pyktok, replicate, requests, selenium, streamlit

### Recent commits (newest first)

- Update README.md
- Update README.md
- Update README.md
- enabled concurrency image processing and included link to article
- run API requests in parallel
- im a prompt engineer
- Merge branch 'main' of https://github.com/inkyant/cruz-hacks-2024
- add video captioning
- edit captions.py
- edit typo on prompt
- edit bug in routes.py
- prompt uses metadata
- edit json to return in routes.py
- add metadata for video
- remove tiktok embed
- improve view when ungenerated
- Merge branch 'main' of https://github.com/inkyant/cruz-hacks-2024
- add snopes fact-check
- edit prompt, print response
- Delete icon, change head title, change content background

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
assemblyai
openai
bs4
lxml 
pyktok
browser-cookie3
numpy
pandas
requests
selenium
streamlit
Flask
Flask-Cors
replicate
opencv-python
cloudinary
```

### client/package.json

```
{
  "name": "client",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "react": "^18.2.0",
    "react-dom": "^18.2.0",
    "react-loader-spinner": "^6.1.6",
    "react-scripts": "5.0.1",
    "react-social-media-embed": "^2.4.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

```

### backend/app.py

```python
from api import start_app

if __name__ == "__main__":
    app = start_app()
    app.run(debug=True)
```

### client/src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

```

### client/src/App.js

```javascript
import logo from './logo.svg';
import './App.css';
import React, { useState } from "react";
// import { TikTokEmbed } from 'react-social-media-embed';
import { RotatingLines } from "react-loader-spinner";

function Loader() {
  return(
    <RotatingLines
      strokeColor="grey"
      strokeWidth='5'
      animationDuration='0.75'
      width='50'
      visible={true}
    />
  )
}

function parseUrl(url) {
  let match = url.match(/https:\/\/www.tiktok.com\/@.+?\/video\/\d+/g)
  console.log(match)
  if (match)
    return match[0]
  else
    return ""
}

function App() {

  const [urlString, setUrlString] = useState("");

  const [infoString, setInfoString] = useState("");
  const [snopesString, setSnopesString] = useState("");
  const [snopesLink, setSnopesLink] = useState("");

  const [loading, setLoading] = useState(false);

  function handleClick(url) {

    setLoading(true)

    fetch("http://localhost:5000/api/fact-check", {
      method: "POST",
      headers: {
      'Content-Type' : 'application/json'
      },
      body: JSON.stringify(url)
    })
    .then(res => res.json())
    .then(data => {
      setInfoString(data.summary)
      setSnopesString(data.snopes)
      setSnopesLink(data.snopes_link)
      setLoading(false)
    })
    .catch(err => console.log(err));
  }

  return (
    <div className="App">
      <head>
          <title>Miss Info</title>
      </head>
      <body>

        <div className="titlebx">
          <h1 className="title">Miss Info</h1>
        </div>

        <div className="wrapper">
          <div className="content" role="main">
          <div className="inbx">
            <input type="text" onChange={(e) => setUrlString(e.target.value)} id="fname" name="fname" value={urlString} placeholder="TikTok Link.."/><br />
            <button onClick={() => handleClick(urlString)} >{loading ? "Loading..." : "Enter"}</button>
          </div>
          {/* <div style={{ display: 'flex', justifyContent: 'center', display: parseUrl(urlString) == "" ? 'none' : '' }}>
            <TikTokEmbed url={parseUrl(urlString)} width={325} />
          </div> */}
          <div className="outbx" style={{display: infoString == "" ? "none" : ""}}>
            {loading ? <Loader/> : <p className="outtxt" id="demo"><h2><b>Results</b></h2>{infoString}</p>}
          </div>
          <div className="outbx" style={{display: snopesString == "" ? "none" : ""}}>
            <h2><b>Related article</b></h2>
            <p className="outtxt">{snopesString}</p>
            <a className="outtxt" href={snopesLink}>{snopesLink}</a>
          </div>
          
          </div>
        </div>
      </body>
    </div>
  );
}

export default App;

```

### backend/api/api_keys.py

```python
OPEN_AI_API_KEY = ""
ASM_AI_API_KEY = ""
```

### backend/api/__init__.py

```python
from flask import Flask
from flask_cors import CORS

def start_app():
    app = Flask(__name__)
    CORS(app)

    from .routes import routes
    app.register_blueprint(routes, url_prefix="/")

    return app
```

### backend/tests/test_send.py

```python
import requests

if __name__ == "__main__":
    api_url = "http://localhost:5000/api/fact-check"
    tiktok_url = "https://www.tiktok.com/t/ZT8pvrCKW/"
    data = {"url": tiktok_url}
    response = requests.post(api_url, data)

    print(response.status_code)
    print(response.json())
```

### client/src/index.css

```css
body {
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

code {
  font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
    monospace;
}

```

### backend/api/asmAI.py

```python
from backend.api.api_keys import ASM_AI_API_KEY
import assemblyai as aai

aai.settings.api_key = ASM_AI_API_KEY

transcriber = aai.Transcriber()
link = "https://v39-eu.tiktokcdn.com/6d4f8b635601721b3f8644f83c11497a/65abbcf6/video/tos/maliva/tos-maliva-ve-0068c799-us/ooEI4dLVdQGfHegATSyAesIsLVIsjuSIqUGWaC/?a=1180&ch=0&cr=13&dr=0&lr=all&cd=0%7C0%7C0%7C&cv=1&br=2740&bt=1370&bti=OHYpOTY0Zik3OjlmOm01MzE6ZDQ0MDo%3D&cs=2&ds=4&ft=XsdJEquYm3FPD12QEH-R3wUIK~c7aeF~O5&mime_type=video_mp4&qs=15&rc=PDpmZzU6OTU0PGhmNGg8ZUBpamg4OHk5cnJlcDMzaTczNEA0MDBgYC0wXmMxLzU0XzA1YSNobGVfMmRzMWlgLS1kMTJzcw%3D%3D&l=20240120062942EC0EB3FA7217861FBDB2&btag=e00095000"
transcript = transcriber.transcribe(link)
print(transcript.text)
```

[7 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]