# Project export: IntervYou This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Insightful interview practice - Devpost: https://devpost.com/software/intervyou - GitHub: https://github.com/DeanShin/CalHacks - Video: https://www.youtube.com/embed/Pod8m1mo7Vo?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 4 GitHub contributor(s) — Dean Shin (53 commits), Nick Ahn (21 commits), Megan (9 commits), Ramon Arambula (1 commits) ## Devpost submission (written by the team) ### Inspiration Interviews can be scary. Whether in person or on zoom, interviewing well takes practice. Communication is the key in any interview, but often interviewees struggle with maintaining open and positive body language while focusing on the quality and content of their answers. Especially for technical interviews, where not only your interpersonal skills are tested, but so is your understanding of various technical concepts. The best interview advice would address the substance the interviewee's answer and their delivery. Because at the heart of it, interviews are a conversation, and no conversation is complete without human expression. ### What it does IntervYou is a self-paced interview practice website. Users can select what kind of interview questions they're preparing for and record themselves answering the question. This video and audio recording is sent to Hume AI, an AI toolkit to understand human emotional expression, and OpenAI's large language model generative pretrained model, ChatGPT. Hume AI's facial expression and speech prosody models will capture and evaluate the nuances of the user's facial movement and vocal expression. IntervYou will then identify parts of the user's answer where they seemed to experience intense emotions that the user can review. In addition, IntervYou sends a transcript of the response (uhhs and ums included!) to ChatGPT for assessment of the accuracy and strength of the user's answer. After answering three questions, users have access to this advice from ChatGPT and emotional insight from HumeAI. ### How we built it Front-End: Reflex using custom Javascript and React components Back-End: Reflex backend using HumeAI API and ChatGPT API ### Challenges we ran into HumeAI Prediction Parsing: Initially, the volume of results generated by the HumeAI batch job was overwhelming. With several layers of nesting were unsure of how to access and best capture the raw data. After several attempts of mapping out the nested data structures and relying on the docs for a baseline understanding of the qualitative significance of the data, we set thresholds for significant data points and saved them in a manageable data structure. Learning a new web development framework Optimizing time cost to handle multiple API requests and process the data Figuring out how to factor both logical and emotional contents from audio-visual components of a video to give feedback to the user ## ### Accomplishments we're proud of Created a multi-threading algorithm for our API data handlers in order to speedup and increase throughput from multiple asynchronous processes Integrating HumeAPI’s sentiment analysis with the logical analysis from ChatGPT to give advice to the interviewee in both soft and hard skills ## ### What we learned How to develop a full-stack web application in Reflex Using multi-threading and asynchronous calls in a webdevelopment environment How to use Sentiment Analysis API and LLM to solve a real-world problem ## ### What's next Improve UI/UX of the website Gather user feedback Extend HumeAI usage to include other insightful models, i.e., Vocal Burst Continue improving the webapp's algorithm for faster and better results ## README (from the GitHub repository) # Welcome to IntervYou! ## Inspiration Interviews can be scary. Whether in person or on zoom, interviewing well takes practice. Communication is the key in any interview, but often interviewees struggle with maintaining open and positive body language while focusing on the quality and content of their answers. Especially for technical interviews, where not only your interpersonal skills are tested, but so is your understanding of various technical concepts. The best interview advice would address the substance the interviewee's answer and their delivery. Because at the heart of it, interviews are a conversation, and no conversation is complete without human expression. ## What it does IntervYou is a self-paced interview practice website. Users can select what kind of interview questions they're preparing for and record themselves answering the question. This video and audio recording is sent to Hume AI, an AI toolkit to understand human emotional expression, and OpenAI's large language model generative pretrained model, ChatGPT. Hume AI's facial expression and speech prosody models will capture and evaluate the nuances of the user's facial movement and vocal expression. IntervYou will then identify parts of the user's answer where they seemed to experience intense emotions that the user can review. In addition, IntervYou sends a transcript of the response (uhhs and ums included!) to ChatGPT for assessment of the accuracy and strength of the user's answer. After answering three questions, users have access to this advice from ChatGPT and emotional insight from HumeAI. ## How we built it Front-End: Reflex using custom Javascript and React components Back-End: Reflex backend using HumeAI API and ChatGPT API ## Challenges we ran into - HumeAI Prediction Parsing: Initially, the volume of results generated by the HumeAI batch job was overwhelming. With several layers of nesting were unsure of how to access and best capture the raw data. After several attempts of mapping out the nested data structures and relying on the docs for a baseline understanding of the qualitative significance of the data, we set thresholds for significant data points and saved them in a manageable data structure. - Learning a new web development framework - Optimizing time cost to handle multiple API requests and process the data - Figuring out how to factor both logical and emotional contents from audio-visual components of a video to give feedback to the user ## Accomplishments that we're proud of - Created a multi-threading algorithm for our API data handlers in order to speedup and increase throughput from multiple asynchronous processes - Integrating HumeAPI’s sentiment analysis with the logical analysis from ChatGPT to give advice to the interviewee in both soft and hard skills ## What we learned - How to develop a full-stack web application in Reflex - Using multi-threading and asynchronous calls in a webdevelopment environment - How to use Sentiment Analysis API and LLM to solve a real-world problem ## What's next for IntervYou * Improve UI/UX of the website * Gather user feedback * Continue improving the webapp's algorithm for faster and better results ## Detected evidence (automated analysis) Indexed codebase: 21 recognized source files, 43 KB. - OpenAI (technology) — detected in the code - Python (language) — detected in the code - JavaScript (language) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (30 of 30) ``` .gitignore calhacks/__init__.py calhacks/advice_raw.txt calhacks/audio_to_advice.py calhacks/calhacks.py calhacks/components/__init__.py calhacks/gpt_client.py calhacks/hume/audio_processing.py calhacks/hume/batch_test.py calhacks/hume/dean_test_data.py calhacks/hume/Dean-Data/negative-emotion-output.json calhacks/hume/Dean-Data/positive-emotion-output.json calhacks/hume/hume_out_parser.py calhacks/hume/outputs/predictions.json calhacks/hume/outputs/sample-audio-and-face.json calhacks/hume/outputs/sample-audio.json calhacks/hume/video_processing.py calhacks/pages/__init__.py calhacks/pages/getstarted.py calhacks/pages/index.py calhacks/pages/quiz.py calhacks/pages/results.py calhacks/pages/setup.py calhacks/questions_raw.txt calhacks/questions.py calhacks/state.py calhacks/styles.py README.md requirements.txt rxconfig.py ``` ### Dependencies - requirements.txt: moviepy, openai, opencv-python@==4.8.1.78, pydub, reflex@==0.3.1, SpeechRecognition ### Recent commits (newest first) - make relative path instead - Fix AI questions - No errors - dean_test_set_emotions: setting emotion rather than str_repr - Added dean set content - Update README - Pushing Dean Data - Prettify - Update README.md - Prettify - Display question alongside modules - Merge branch 'main' of github.com:DeanShin/CalHacks - Resolve merge conflicts - Merge pull request #18 from DeanShin/feature/fully-integrate - Fully connect with mock data - Move hume files and fix video rendering - move hume files - Merge branch 'main' of github.com:DeanShin/CalHacks - Remove unused import - Change hume parsing logic to get top overall average and highs ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` reflex==0.3.1 opencv-python==4.8.1.78 moviepy SpeechRecognition pydub openai ``` ### calhacks/pages/index.py ```python """The dashboard page.""" import reflex as rx @rx.page(route="/") def index() -> rx.Component: return rx.grid( rx.box(), rx.heading( rx.span( "IntervYou", display="content-block", padding=".1em .1em", border_radius="1000px", background="#BFDBF7" ), size="2xl", text_align="center"), rx.heading('AI-driven interview preparation for self-improvement', size="md", text_align="center"), rx.text('a project by Memory Leeks, Calhacks 10.0', text_align="center"), rx.center( rx.link( rx.button('Get Started', background="#BFDBF7"), href='http://localhost:3000/setup', ), ), rx.box(), bg='white', width='100dvw', height='100dvh', align_items="end", grid_template_rows='1fr 20% 10% 10% 20% 2fr' ) ``` ### rxconfig.py ```python import reflex as rx config = rx.Config( app_name="calhacks", ) ``` ### calhacks/__init__.py ```python """Base template for Reflex.""" ``` ### calhacks/styles.py ```python """Styles for the app.""" import reflex as rx base_style = { "box_sizing": "border-box", "margin": "0" } ``` ### calhacks/calhacks.py ```python """Welcome to Reflex!.""" from calhacks import styles # Import all the pages. from calhacks.pages import * import reflex as rx # Create the app and compile it. app = rx.App(style=styles.base_style) app.add_page(index, '/') app.compile() ``` ### calhacks/state.py ```python """Base state for the app.""" import reflex as rx class State(rx.State): """Base state for the app. The base state is used to store general vars used throughout the app. """ interview_context: str questions = [] pass ``` ### calhacks/audio_to_advice.py ```python """This file will take in a webp audio file, process it, build and send a gpt query, and save the response """ # TODO: async something to work with Dean's results.py from gpt-clint import ChatGPT from audio_processing.py import convert_video_to_wav, wav_to_audio_transcript import asyncio def main(question: str, file: str): -> str # process file audio_processor = Video() audio_processor.convert_video_to_wav(file) transcript = audio_processor.wav_to_audio_transcript(file) # make gpt client instance gpt = ChatGPT(question, transcript) asyncio.run(gpt.get_advice()) return gpt.advice ``` ### calhacks/questions.py ```python role_to_questions = { "Behavioral": [ "Tell me about a time you worked in a team and dealt with an issue", "Tell me about a time you were under a lot of pressure at work or at school. What was going on, and how did you get through it?", "Tell me about your proudest professional accomplishment", "Tell me about a time you had to tell someone no.", "Do you have any work habits you’d like to improve?", ], "Front-end": [ "What do you do to ensure a site is user friendly?", "What techniques do you use to improve a site’s performance?", "Describe three ways to decrease page load time", "Suggest some ways on how to fix the browser-specific styling issue", "What are pseudo-classes? Provide a few real-world use cases", ], "Back-end": [ "Why would you opt for Mircroservice Architecture?", "What are the seven layers of the OSI system model?", "If you have a limited amount of memory, how would you handle a large amount of data?", "What steps would you take to use mysqldump to restore MySQL?", "How can you build scalability into a software program?", "What is containerization?" ], "Full-stack": [ "How would you find a memory leak?", "How would you debug and resolve a site issue?", "What strategies would you pick up for measuring the success of a new feature?", "What is responsive design? What are its best practices?", "What do event capturing and event bubbling have in common?", "What is the difference between acceptance and functional testing?", ], "AI/ML": [ "What is overfitting? How can you avoid It?", "What is ‘training Set’ and ‘test Set’ in a machine learning model? How much data will you allocate for Your training, validation, and test sets?", "What is the difference between inductive machine learning and deductive machine learning? Give an example for both", "Describe a situation where you would use classification over regression", "Describe the difference between unsupervised learning and supervised learning?" ], "Devops": [ "Describe the different phases on DevOps", "What is the difference between a centralized and distributed version control system(VCS)?", "Can you recall a time when a DevOps project was unsuccessful? Why was it unsuccessful?", "What are some strategies to ensure high availability and fault tolerance in a distributed system?", "Describe your process for automating testing.", ] } ``` ### calhacks/gpt_client.py ```python import os import asyncio import openai if not os.environ.get("OPENAI_API_KEY"): raise Exception("OpenAI api key not set") class ChatGPT: def __init__(self, user_context: str, user_answer: str): """ Parameters ---------- user_context : str Frontend input that lets user define scope of interview questions """ self.user_context = user_context self.count = 0 self.questions = list[str] self.advice = "" self.user_answer = "" async def _generate_questions(self) -> dict: print('\t-generating questions') prompt = f"""Generate me a list of 10 technical interview questions. 5 of which will focus on {self.user_context}. The remaining 5 will be behavioural. Just output the list numbered 1-10. Do not include labels. Do not a response verifying my message""" # Prompt chatgpt response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[ { "role": "system", "content": f"You are a helpful interviewer that will help me prep for my {self.user_context} interview", }, {"role": "user", "content": f"{prompt}"}, ], ) # return response["choices"][0]["message"]["content"] return response def _parse_questions(self, questions_raw: dict) -> list[str]: print('\t-parsing questions') raw = questions_raw["choices"][0]["message"]["content"] #with open('questions_raw.txt', 'a+') as f: # f.write(raw) # f.write("\n") return raw.split("\n") async def set_questions(self) -> None: print('\t-set questions') questions = await self._generate_questions() self.questions = self._parse_questions(questions) async def _generate_advice(self) -> str: print('\t-generating advice') prompt = f"""I was asked as an interviewee {self.user_context} And I responded with {self.user_answer}. What advice do you have for me?""" # prompt chatgpt advice = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=[ { "role": "system", "content": f"You will act as a coding interview coach that gives concise advice." }, {"role": "user", "content": f"{prompt}"}, ], ) return advice def _parse_advice(self, advice_raw: dict) -> list[str]: print('\t-parsing advice') raw = advice_raw["choices"][0]["message"]["content"] with open('advice_raw.txt', 'a+') as f: f.write(raw) return raw async def get_advice(self) -> None: print('\t-get advice') advice = await self._generate_advice() self.advice = self._parse_advice(advice) """ async def main(): client = ChatGPT("What is chmod in Linux?") #questions = await client.set_questions() #print(client.questions) advice = await client.get_advice() print(client.advice) if __name__ == "__main__": asyncio.run(main()) """ ``` [10 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]