# Project export: Yujafiy

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: CruzHacks 2024
- Tagline: Your Lecture, Your Way!
- Devpost: https://devpost.com/software/yujafy
- GitHub: https://github.com/HershR/yujafy_backend
- Demo: https://github.com/ttoy12/Yujafiy
- Video: https://www.youtube.com/embed/WXHbGDdesUI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — Hersh (16 commits), e94904 (6 commits)

## Devpost submission (written by the team)

### Inspiration

Often, I find myself facing great difficulty staying focused and engaged watching Yuja, our recorded lectures provider. Some of my professors do not speak very clearly and I can only listen to the same person speak for so long. With the advent of AI voice generation technologies we wondered if it could be possible to replace monotonous voices with voices that we love and pay great attention to direct our lecture recordings.

### What it does

The extension uses artificial intelligence to transcribe the video lecture and uses an AI voice model that are modeled after very famous and recognizable characters such as Mickey Mouse, Donald Trump, Morgan Freeman, any many more to come to make yuja lectures that more engaging.

### How we built it

We started by making a Chrome extension using key technologies such as HTML, CSS and Javascript inside of the extension. On the backend side of things, we used flask and Python to create our backend server to communicate with our extension. We also utilized the free API provided by Topmediai.com to convert our transcripts into fun and exiting voices at are then returned to the backend and then returned back to the frontend and onto the user.

### Challenges we ran into

One major problem we had was the fact we weren't able to scrape the transcript from the Yuja page and and send it off straight to our backend. Since Yuja is a private domain, it obfuscates the elements that we needed. In the end, we resorted to the user saving the transcript then uploading into our extension. Another challenge was figuring out how to use the AI model that we found as the instructions provided weren't very clear. Another challenge was figuring out the flask server and configuring it. Another major challenge was building the extension itself as none of us had ever built a chrome extension in our life.

### Accomplishments we're proud of

We are very proud to get the front end and the back end communicating effectively as that was one major hurdle we had to figure our as we had never done it before. We also are very proud of figuring out how to use the voice API service as we also had never used one before and the instructions provided were not the best. We are very pleased with how the extension looks. But most importantly, we are very proud that we got it all working.

### What we learned

Some things are a lot harder than they seem. We also learned that we can't expect everything to work the first time. But it is also very rewarding when we solve these hard problems. My partners also learned how to use Git, GitHub desktop, and learn some new Javascript.

### What's next

for Yujafy We are planning to integrate translation features to translate the lectures in it's entirely so non native english speakers can also enjoy and understand the pre recorded video lectures. We are also planning to integrate a lecture summary feature and possibly a quiz feature based on topics found in the lecture.

## README (from the GitHub repository)

# yujafiy_backend
Backend of for tts Yuja videos
Currently made to be run locally

# Set Up
1. install packages in requirments.txt with "pip install -r requirements.txt"
2. get API key from [topmediai.com](https://www.topmediai.com/api/text-to-speech-api/)
3. in ./flaskr/api.py replace API key var
4. run server with 
```
$ python run.py
```
5. Server should start, if you have issue with using port 5000, edit the port in run.py and all api calls in the frontend extension

# Cache
All processed videos are saved and can be retrieved
Audio file url are stored in ./audiofiles. Each file uses the following naming convention "videoPid_voiceId.json"
Each file stores a list of urls. 



## Detected evidence (automated analysis)

Indexed codebase: 10 recognized source files, 10 KB.
- Python (language) — detected in the code
- CSS (language) — claimed on Devpost, not found in the code
- Flask (technology) — claimed on Devpost, not found in the code
- JavaScript (language) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (14 of 14)

```
.gitignore
all_voice_ids.json
audiofiles/1139851_00152639-3826-11ee-a861-00163e2ac61b.json
config.py
demo scripts and misc/get_alll_possible_voices.py
demo scripts and misc/get_api_info.py
demo scripts and misc/testing.py
demo scripts and misc/voicetest.py
flaskr/__init__.py
flaskr/api.py
flaskr/request_utils.py
README.md
requirments.txt
run.py
```

### Dependencies

No dependency index available.

### Recent commits (newest first)

- organized the folder structure better
- old voice files and new api key
- Update README.md
- Revert "Update voicetest.py"
- Update voicetest.py
- Update README.md
- updated API key and retrieve func
- Merge pull request #1 from HershR/timestamp-parsing
- created process json api call
- Update api.py
- api works with cors
- added test script
- Merge branch 'main' of github.com:HershR/yujafy_backend
- able to save links to files and retrieve them
- literally all the voice ids to pick from
- process func implemented
- backend setup and working
- updated voice speaker
- voice test works
- renamed file

## Key source files (fetched from GitHub, selected and truncated for size)

### config.py

```python
class Config:
    SECRET_KEY = "SECRETKEY123"
    DEBUG = True
    API_KEY = "1b547580cfd44b27b1647aec0fafcddc"
    # FILE_PATH = "./audiofiles"

```

### run.py

```python
from flaskr import create_app
from config import *

if __name__ == "__main__":
    # app = create_app(config=config.TestingConfig)  # Dockerfile should use python run.py
    app = create_app()
    app.run(host='0.0.0.0', port=5000)

```

### flaskr/__init__.py

```python
import os
from flask import Flask


def create_app(config='config.Config'):
    app = Flask(__name__)
    app.config.from_object(config)
    print(f'created app with config: {config}')

    @app.route('/')
    def hello():
        return 'Hello, World!'

    from . import api
    app.register_blueprint(api.bp)

    return app

```

### demo scripts and misc/get_alll_possible_voices.py

```python
import requests

url = "https://api.topmediai.com/v1/voices_list"
headers = {
    "accept": "application/json",
    "x-api-key": "1b547580cfd44b27b1647aec0fafcddc"
}

response = requests.get(url, headers=headers)

# The response body is a JSON string, so we parse it into a Python dictionary
data = response.json()

# Now, data is a Python dictionary that contains the response body
print(data)


```

### demo scripts and misc/get_api_info.py

```python
import requests

url = "https://api.topmediai.com/v1/get_api_key_info"
headers = {
    "accept": "application/json",
    "x-api-key": "1b547580cfd44b27b1647aec0fafcddc"
}

response = requests.get(url, headers=headers)

# The response body is a JSON string, so we parse it into a Python dictionary
data = response.json()

# Now, data is a Python dictionary that contains the response body
print(data)

```

### demo scripts and misc/testing.py

```python
from pprint import pprint

import requests as requests

s = "With these changes, " \
    "your extension will display a " \
    "popup with a button, and when the" \
    " button is pressed, it will make " \
    "the specified request using the fetch " \
    "API. Adjust the code according to your " \
    "specific requirements and extension design."

def process():
    url = 'http://localhost:5000/api/process'
    param = \
        {
            'id': 'test',
            'sentences': [s],
            'voice': 'default'
        }

    r = requests.post(url, json=param)
    if r.status_code == 200:
        print(r.json())
    else:
        print('failed')
        pprint(r)


def retrieve():
    url = 'http://localhost:5000/api/retrieve?v_id=test'
    r = requests.get(url)
    if r.status_code == 200:
        print(r.json())
    else:
        print('failed')
        pprint(r)


if __name__ == "__main__":
    process()

```

### flaskr/request_utils.py

```python
from flask import request

# ARGS
ID = 'video_id'
SENTENCES = 'sentences'
VOICE = 'voice'
ENTRIES = 'entries'


def check_get_args(args_list):
    """
    Make sure all required args are in get
    :return: True, None if OK, False, missing arg
    """
    missing_args = [arg for arg in args_list if not request.args.get(arg, False)]
    if len(missing_args) > 0:
        return False, {"missing parameters": str(missing_args)}
    return True, None


def check_get_form_args(args_list):
    missing_args = [arg for arg in args_list if not request.form.get(arg, False)]
    if len(missing_args) > 0:
        return False, {"missing parameters": str(missing_args)}
    return True, None


def check_json_post_args(args_list):
    """
    Make sure all required args are keys in posted json
    :return: True, None if OK, False, missing args
    """
    missing_args = [arg for arg in args_list if arg not in request.json.keys()]
    if len(missing_args) > 0:
        return {}, {"missing parameters": str(missing_args)}
    return True, None

```

### demo scripts and misc/voicetest.py

```python
import requests
import json
import textwrap

def text_to_speech():
    url = "https://api.topmediai.com/v1/text2speech"
    headers = {
        "accept": "application/json",
        "x-api-key": "1b547580cfd44b27b1647aec0fafcddc",
        "Content-Type": "application/json"
    }

    # Get user input
    text = input("Enter the text you want to convert to speech: ")

    # Split the text into chunks of up to 250 characters without splitting words
    chunks = textwrap.wrap(text, width=250)

    urls = []

    for chunk in chunks:
        data = {
          "text": chunk,
          "speaker": "0015548d-3826-11ee-a861-00163e2ac61b",
          "emotion": "Neutral"
        }

        response = requests.post(url, headers=headers, data=json.dumps(data))

        # The response body is a JSON string, so we parse it into a Python dictionary
        data = response.json()

        # Check if 'data' is in the response
        if 'data' in data:
            # Print the oss_url to the screen
            print(data['data']['oss_url'])

            # Add the oss_url to the list
            urls.append(data['data']['oss_url'])
        else:
            print("Error: 'data' not in response")

    # Return the list of oss_urls
    return urls

# Call the function
print(text_to_speech())

```

### flaskr/api.py

```python
from flask import Blueprint, request, jsonify, current_app
from flask_cors import CORS, cross_origin
from .request_utils import ID, ENTRIES, SENTENCES, VOICE, check_json_post_args, check_get_args
import requests
import json

bp = Blueprint('api', __name__)

API_KEY = "8d98dac6214a4732bdbce3b44447810a"

FILE_PATH = "./audiofiles"

DEFAULT_VOICE = "0015548d-3826-11ee-a861-00163e2ac61b"


@bp.route('/api/process', methods=['POST'])
@cross_origin()
def process_text():
    # write up def
    """
    :param: sentences: List , voice: AI voice
    :return:
    """
    _, error = check_json_post_args([SENTENCES, VOICE])
    if error:
        return jsonify(error), 400
    url = "https://api.topmediai.com/v1/text2speech"
    v_id = request.json.get(ID)
    sentences = request.json.get(SENTENCES)
    voice_id = DEFAULT_VOICE if request.json.get(VOICE) == 'default' else request.json.get(VOICE)
    print(sentences, voice_id)
    headers = {
        "accept": "application/json",
        "x-api-key": API_KEY,
        "Content-Type": "application/json"
    }
    audio_links = []
    for sentence in sentences:
        max_char_size = 200
        chunks = split_string(sentence, max_char_size)

        # links = []

        for chunk in chunks:
            param = {
                "text": chunk.replace('\ufeff', ''),
                "speaker": voice_id,
                "emotion": "Neutral"
            }

            r = requests.post(url, headers=headers, data=json.dumps(param))

            # The response body is a JSON string, so we parse it into a Python dictionary
            if r.status_code == 200:
                data = r.json()
                try:
                    print(chunk, data['data']['oss_url'])
                    audio_links.append(data['data']['oss_url'])
                except BaseException as e:
                    print(e)
                    print(chunk)
                    print('data', data)
            else:
                print(r)
                # links.append(None)
        # audio_links[len(audio_links)] = links
    file_name = f'{FILE_PATH}{v_id}_{voice_id}.json'

    # import os
    # current_directory = os.getcwd()
    # print(f"The current working directory is: {current_directory}")

    with open(file_name, 'w') as json_file:
        json.dump(audio_links, json_file, indent=4)
    return jsonify({'data': audio_links}), 200


@bp.route('/api/processjson', methods=['POST'])
@cross_origin()
def process_json():
    '''
    :param: video_id: int, entries: List [{index, sentence, timestart, timeend}] , voice: AI voice
    :return:
    '''
    _, error = check_json_post_args([ID, ENTRIES, VOICE])
    if error:
        return jsonify(error), 400
    url = "https://api.topmediai.com/v1/text2speech"
    v_id = request.json.get(ID)
    entries = request.json.get(ENTRIES)
    voice_id = DEFAULT_VOICE if request.json.get(VOICE) == 'default' else request.json.get(VOICE)
    headers = {
        "accept": "application/json",
        "x-api-key": API_KEY,
        "Content-Type": "application/json"
    }
    audio_links = {}
    for entry in entries:
        index = entry['index']
        sentence = entry['sentence']
        #timeStart = entry['timeStart']
        #timeEnd = entry['timeEnd']

        chunks = split_string(sentence, 250)

        links = []

        for chunk in chunks:
            param = {
                "text": chunk.replace('\ufeff', ''),
                "speaker": voice_id,
                "emotion": "Neutral"
            }

            r = requests.post(url, headers=headers, data=json.dumps(param))

            # The response body is a JSON string, so we parse it into a Python dictionary
            if r.status_code == 200:
                data = r.json()
                try:
                    #print(chunk, data['data']['oss_url'])
                    links.append(data['data']['oss_url'])
                except BaseException as e:
                    print(e)
                    print(chunk)
                    #print('data', data)
            else:
                print(r)
                # links.append(None)
        audio_links[index] = links
    file_name = f'{FILE_PATH}\{v_id}_{voice_id}.json'

    with open(file_name, 'w') as json_file:
        json.dump(audio_links, json_file, indent=4)

    return jsonify({'data': audio_links}), 200


@bp.route('/api/retrieve', methods=['GET'])
@cross_origin()
def retrieve_audio():
    # write up def
    """
    :param: v_id: video id, voice_id
    :return:
    """
    v_id = request.args.get(ID, default='none', type=str)
    voice_id = request.args.get(VOICE, default='none', type=str)
    print(v_id)
    # else work with data
    file_path = f"{FILE_PATH}/{v_id}_{voice_id}.json"
    try:
        with open(file_path, 'r') as json_file:
            audio_links = json.load(json_file)
    except BaseException as e:
        print('error', e)
        return jsonify("error: Unable to find file audio files corresponding to ID"), 404
    print({'data': audio_links})
    return jsonify({'data': audio_links}), 200


# /v1/voices_list

def split_string(input_string, max_length):
    result = []

    for i in range(0, len(input_string), max_length):
        result.append(input_string[i:i + max_length])

    return result

```