# Project export: Impromptu This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Learn to think on the spot. - Devpost: https://devpost.com/software/impromptu-y1qfds - GitHub: https://github.com/Nazarshia2889/Impromptu - Team: 4 GitHub contributor(s) — Nathan Dai (14 commits), Ritvik Irigireddy (10 commits), Arshia Nayebnazar (4 commits), therealshreyas (1 commits) ## Devpost submission (written by the team) ### Inspiration Impromptu speaking is a critical skill in many areas of life, from high-pressure job interviews to persuading clients or defending arguments in live debates. Despite its importance, practicing impromptu speaking is notoriously difficult. The most effective way to improve is through real verbal conversations, but finding people to practice with who can also provide valuable critiques is rarely convenient or feasible. ### What it does Impromptu leverages speech-to-speech technology to help users practice thinking and speaking on the spot. The app starts by presenting a debate topic or question related to real-world challenges, prompting users to create an initial argument. Users have a limited time to jot down notes for their speech, and if they need inspiration, our Gemini-powered assistant offers alternative perspectives or questions to consider. Once preparation is complete, users enter a virtual room with three different AI critics, each possessing unique voices and personalities. Users present their prepared speech, and afterward, they engage in a live debate with the critics, answering questions and defending their arguments. Impromptu goes beyond traditional speaking practice by offering dynamic, real-time feedback from intelligent models, helping users refine their ability to form persuasive arguments on the spot and think quickly under pressure. ### How we built it The app’s debate topics are generated by Google’s Gemini API, ensuring high-quality prompts using in-context learning. The Gemini-powered assistant also aids users in brainstorming ideas. Once a user presents their speech, Groq’s Whisper Large V3 Turbo transcribes it to text. For the debate portion, we use the VAPI API to set up speech-to-speech interactions with three distinct AI critics. Each model challenges key ideas, encouraging the user to strengthen their arguments quickly. The entire web app runs on Next.js with Tailwind CSS for seamless user experience and design. ### Challenges we ran into One of the biggest hurdles we encountered was coordinating a smooth conversation between multiple AI voice models and the user. Establishing a one-on-one conversation through the VAPI API was straightforward, but handling simultaneous interactions between the user and multiple critics was challenging. Voice overlap and inconsistent responses were common issues. To solve this, we implemented a system where each model received the entire conversation history, including the transcription of the user’s speech, ensuring consistency in critiques without voice conflicts. Another challenge was optimizing the models' feedback quality, especially during the initial speech when interruptions could disrupt the user’s flow. To mitigate this, we used transcription to capture the speech-to-text conversion, providing the models with a coherent basis for their responses during the live conversation. Additionally, we worked to convert audio packets to transcribed text using the computer's microphone, integrating it smoothly into the app’s interface. ### Accomplishments we're proud of We are proud of successfully integrating multiple AI assistants to provide users with varied and constructive feedback based on their speech. Implementing a working timer and audio recording system was another highlight, ensuring that users' timing requests and speeches were captured accurately. Additionally, we created an intuitive landing page that allows users to learn more about our product and dive straight into practice. ### What we learned This project taught us how to integrate various APIs into a cohesive, functional system. We gained hands-on experience with VAPI, Deepgram, Groq, and Gemini, expanding our knowledge of state-of-the-art AI tools. On the non-technical side, we improved our ability to divide and conquer complex problems, enhancing team collaboration. ### What's next Looking ahead, we plan to introduce several new features, such as customizable levels of critique from the AI assistants, a leaderboard to foster competitiveness and encourage daily practice, and the option to generate article links for users seeking research guidance on debate topics. ## README (from the GitHub repository) This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app). ## Getting Started First, run the development server: ```bash npm run dev # or yarn dev # or pnpm dev # or bun dev ``` Open [http://localhost:3000](http://localhost:3000) with your browser to see the result. You can start editing the page by modifying `app/page.js`. The page auto-updates as you edit the file. This project uses [`next/font`](https://nextjs.org/docs/basic-features/font-optimization) to automatically optimize and load Inter, a custom Google Font. ## Learn More To learn more about Next.js, take a look at the following resources: - [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API. - [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial. You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js/) - your feedback and contributions are welcome! ## Deploy on Vercel The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js. Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details. ## Detected evidence (automated analysis) Indexed codebase: 29 recognized source files, 54 KB. - CSS (language) — detected in the code - Google Gemini (technology) — detected in the code - JavaScript (language) — detected in the code - LangChain (technology) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code ## Codebase structure (from repository index) ### Files (36 of 36) ``` .eslintrc.json .gitignore app/globals.css app/home/page.js app/judge/page.js app/layout.js app/notes/page.js app/page.js app/score/page.js components/notes/NotesEditor.js components/notes/Suggestions.js components/notes/TopicDisplay.js components/recording/JudgesFeedback.js components/recording/RecordingControls.js components/recording/RecordingSidebar.js components/topic/TimingSection.js components/topic/TopicSection.js components/ui/Button.js components/ui/InputField.js components/ui/SelectField.js components/ui/Timer.js data/questions.js jsconfig.json next.config.mjs package.json postcss.config.mjs README.md scripts/generate_topic.py scripts/prompts/generate_topic.txt tailwind.config.js utils/apiUtils.js utils/audioHandlers.js utils/cryptography.js utils/noteSuggestion.js utils/prompt.js utils/recordingHandlers.js ``` ### Dependencies - package.json: @deepgram/sdk@^3.8.1, @google/generative-ai@^0.21.0, @langchain/core@^0.3.18, @langchain/groq@^0.1.2, @langchain/langgraph@^0.2.20, @openai/realtime-api-beta@github:openai/openai-realtime-api-beta, @vapi-ai/web@^2.1.4, axios@^1.7.7, buffer@^6.0.3, crypto-js@^4.2.0, dotenv@^16.4.5, eslint@^8, eslint-config-next@14.2.15, groq-sdk@^0.7.0, next@14.2.15, openai@^4.68.1, postcss@^8, process@^0.11.10, react@^18, react-dom@^18, react-slick@^0.30.2, slick-carousel@^1.8.1, tailwindcss@^3.4.1, uuid@^11.0.3, web-audio-api@^0.2.2, webpack@^5.96.1, webpack-cli@^5.1.4 ### Recent commits (newest first) - judging room - feat: add playAudioBrowser function to handle audio playback in the browser - feat: add topic validation logic and update button state based on topic validity - refactor: reorganize imports in judge page for improved readability and structure - refactor: streamline audio handler functions and improve import structure - feat: refactor recording page and audio handling logic; implement new audio handlers and improve timer functionality - feat: add recording controls, judges feedback, and sidebar components to enhance recording page functionality - feat: implement API utility functions and initialize APIs in RecordingPage - refactor: integrate Timer component and optimize recording logic - refactor: extract judge feedback prompt and implement decryption utility - empty test - add transcript state and update localStorage handling for speech history - flip timing and topic selection - sus - no leaked api keys - fix redirect to score - minor changes to judges - fix timer - package.json fixes - Update page.js ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "impromptu", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@deepgram/sdk": "^3.8.1", "@google/generative-ai": "^0.21.0", "@langchain/core": "^0.3.18", "@langchain/groq": "^0.1.2", "@langchain/langgraph": "^0.2.20", "@openai/realtime-api-beta": "github:openai/openai-realtime-api-beta", "@vapi-ai/web": "^2.1.4", "axios": "^1.7.7", "crypto-js": "^4.2.0", "dotenv": "^16.4.5", "groq-sdk": "^0.7.0", "next": "14.2.15", "openai": "^4.68.1", "react": "^18", "react-dom": "^18", "react-slick": "^0.30.2", "slick-carousel": "^1.8.1", "uuid": "^11.0.3", "web-audio-api": "^0.2.2" }, "devDependencies": { "buffer": "^6.0.3", "eslint": "^8", "eslint-config-next": "14.2.15", "postcss": "^8", "process": "^0.11.10", "tailwindcss": "^3.4.1", "webpack": "^5.96.1", "webpack-cli": "^5.1.4" } } ``` ### app/layout.js ```javascript import localFont from 'next/font/local'; import './globals.css'; const geistSans = localFont({ src: '../public/fonts/GeistVF.woff', variable: '--font-geist-sans', weight: '100 900', }); const geistMono = localFont({ src: '../public/fonts/GeistMonoVF.woff', variable: '--font-geist-mono', weight: '100 900', }); export const metadata = { title: 'Create Next App', description: 'Generated by create next app', }; export default function RootLayout({ children }) { return ( {children} ); } ``` ### app/page.js ```javascript 'use client'; import { useState } from 'react'; import { useRouter } from 'next/navigation'; import TopicSection from '@/components/topic/TopicSection'; import TimingSection from '@/components/topic/TimingSection'; const TopicGenerator = () => { const [step, setStep] = useState(1); const [prepTime, setPrepTime] = useState('30'); const [speakingTime, setSpeakingTime] = useState('120'); const router = useRouter(); const handleNext = () => { setStep(2); }; const handleConfirm = () => { localStorage.setItem('prepTime', prepTime); localStorage.setItem('speakingLength', speakingTime); router.push('/notes'); }; return (
{step === 1 && ( )} {step === 2 && }
); }; export default TopicGenerator; ``` ### app/notes/page.js ```javascript 'use client'; import { useEffect, useState, useRef, useCallback } from 'react'; import { useRouter } from 'next/navigation'; import Timer from '@/components/ui/Timer'; import Suggestions from '@/components/notes/Suggestions'; import NotesEditor from '@/components/notes/NotesEditor'; import TopicDisplay from '@/components/notes/TopicDisplay'; const NotesPage = () => { const [topic, setTopic] = useState(''); const [prepTime, setPrepTime] = useState(null); // Set to null initially to indicate it's not ready yet const [notes, setNotes] = useState(''); const notesRef = useRef(notes); const router = useRouter(); // Update notesRef whenever notes change useEffect(() => { notesRef.current = notes; }, [notes]); // Memoize handleTimerEnd so it doesn't change on every render const handleTimerEnd = useCallback(() => { localStorage.setItem('notes', notesRef.current); localStorage.setItem('geminiSuggestions', JSON.stringify([])); router.push('/judge'); // Redirect to /judge route when timer reaches 0 }, [router]); // Load initial data for topic and prep time useEffect(() => { const savedTopic = localStorage.getItem('topic'); const prepTimeValue = parseInt(localStorage.getItem('prepTime'), 10); // Parse to ensure it's a number if (savedTopic && !isNaN(prepTimeValue)) { setTopic(savedTopic); setPrepTime(prepTimeValue); } }, []); return (
{prepTime !== null && ( // Render only after prepTime is loaded <> {/* Timer Wrapper to align Timer to the right */}
)}
); }; export default NotesPage; ``` ### app/score/page.js ```javascript 'use client'; import { useEffect, useState } from 'react'; import { getScorePrompt, groqStructureToolsParameters } from '@/utils/prompt'; import Groq from 'groq-sdk'; const Score = () => { const [history, setHistory] = useState(''); const [topic, setTopic] = useState(''); const [score, setScore] = useState({}); const groq = new Groq({ apiKey: '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', dangerouslyAllowBrowser: true, }); // Fetch and set topic and history from localStorage useEffect(() => { const storedHistory = localStorage.getItem('transcript'); const storedTopic = localStorage.getItem('topic'); if (storedTopic) { setTopic(storedTopic); } if (storedHistory) { setHistory(storedHistory); } }, []); // Run getScore only when topic and history are set useEffect(() => { if (topic && history) { getScore(); } }, [topic, history]); // Define async function for Groq API call with structured output const getGroqChatCompletion = async (prompt) => { return await groq.chat.completions.create({ model: 'llama3-groq-70b-8192-tool-use-preview', messages: [ { role: 'user', content: prompt, }, ], tools: groqStructureToolsParameters, tool_choice: 'auto', }); }; // Fetch score based on topic and history const getScore = async () => { try { console.log('topic:', topic); console.log('history:', history); const prompt = getScorePrompt(topic, history); const result = await getGroqChatCompletion(prompt); setScore(JSON.parse(result.choices[0]?.message?.content) || {}); } catch (error) { console.error('Error getting score:', error); } }; console.log(score); const categories = Object.keys(score); return (

Speech Evaluation Summary

{categories.map((category) => (

{category.replace(/_/g, ' ')}

Score: {score[category].score} / 10

{score[category].justification}

))}
); }; export default Score; ``` ### app/judge/page.js ```javascript 'use client'; import { useEffect, useRef, useState } from 'react'; import { Inter } from 'next/font/google'; import dotenv from 'dotenv'; import JudgesFeedback from '@/components/recording/JudgesFeedback'; import RecordingControls from '@/components/recording/RecordingControls'; import RecordingSidebar from '@/components/recording/RecordingSidebar'; import { getAndPlayAudio, getResponse, handleTranscribeAudio } from '@/utils/audioHandlers'; import { initializeAPIs } from '@/utils/apiUtils'; import { getJudgeFeedbackContextPrompt } from '@/utils/prompt'; import { startMicrophoneStream, stopRecording } from '@/utils/recordingHandlers'; dotenv.config(); const inter = Inter({ subsets: ['latin'] }); const RecordingPage = () => { // State const [notes, setNotes] = useState(''); const [isRecording, setIsRecording] = useState(false); const [hasStarted, setHasStarted] = useState(false); const [speakingLength, setSpeakingLength] = useState(0); const [suggestions, setSuggestions] = useState([]); const [topic, setTopic] = useState(''); const [currentSpeaker, setCurrentSpeaker] = useState(''); const [transcript, setTranscript] = useState(''); // Refs const mediaRecorder = useRef(null); const stream = useRef(null); const history = [ { role: 'system', content: getJudgeFeedbackContextPrompt(topic), }, ]; // Initialize APIs const APIs = initializeAPIs(); const { transcribeGroq, groq, openai } = APIs; // Load initial data useEffect(() => { const storedNotes = localStorage.getItem('notes'); const storedSpeakingLength = localStorage.getItem('speakingLength'); const storedSuggestions = localStorage.getItem('geminiSuggestions'); const storedTopic = localStorage.getItem('topic'); if (storedNotes) setNotes(storedNotes); if (storedSpeakingLength) setSpeakingLength(parseInt(storedSpeakingLength, 10)); if (storedSuggestions) setSuggestions(JSON.parse(storedSuggestions)); if (storedTopic) setTopic(storedTopic); }, []); // Create bound versions of handlers that include necessary state/props const boundGetAndPlayAudio = async (responseText) => { await getAndPlayAudio(responseText, openai, setCurrentSpeaker); }; const boundGetResponse = async (userText) => { await getResponse(userText, groq, history, setTranscript, boundGetAndPlayAudio); }; const boundHandleTranscribeAudio = async (file) => { await handleTranscribeAudio(file, transcribeGroq, boundGetResponse); }; // Event Handlers const handleTimerEnd = () => { if (isRecording) { handleStopRecording(); } }; const handleStartRecording = () => { setIsRecording(true); setHasStarted(true); startMicrophoneStream(mediaRecorder, stream, boundHandleTranscribeAudio); setCurrentSpeaker('User'); }; const handleStopRecording = () => stopRecording(isRecording, mediaRecorder, stream, setIsRecording); return (
{/* Left Section (Recording + Judges) */}
{/* Current Speaker Box */}

Speaking: {currentSpeaker}

); }; export default RecordingPage; ``` ### app/home/page.js ```javascript 'use client'; import { useEffect } from 'react'; import Slider from 'react-slick'; import 'slick-carousel/slick/slick.css'; import 'slick-carousel/slick/slick-theme.css'; const LandingPage = () => { useEffect(() => { const svgElement = document.getElementById('background-svg'); if (svgElement) { const elements = svgElement.querySelectorAll('polygon'); // Function to make each hexagon move smoothly and randomly elements.forEach((element) => { const randomMovement = () => { const moveX = Math.random() * 4000 - 2000; // Larger screen simulation const moveY = Math.random() * 3000 - 1500; // Larger screen simulation const duration = 20000 + Math.random() * 10000; // Duration between 15-20 seconds to slow down the fastest hexagons element.animate( [ { transform: element.style.transform }, { transform: `translate(${moveX}px, ${moveY}px) rotate(${element.dataset.rotation}deg)`, }, ], { duration: duration, // Slower movement easing: 'ease-in-out', iterations: Infinity, } ); }; randomMovement(); }); } // Add smooth scrolling behavior const smoothScroll = (e) => { e.preventDefault(); const targetId = e.currentTarget.getAttribute('href'); const targetElement = document.querySelector(targetId); if (targetElement) { targetElement.scrollIntoView({ behavior: 'smooth' }); } }; const learnMoreButton = document.querySelector('#learn-more-button'); if (learnMoreButton) { learnMoreButton.addEventListener('click', smoothScroll); } // Clean up event listener return () => { if (learnMoreButton) { learnMoreButton.removeEventListener('click', smoothScroll); } }; }, []); const carouselSettings = { dots: true, infinite: true, speed: 500, slidesToShow: 1, slidesToScroll: 1, autoplay: true, autoplaySpeed: 5000, }; return (
Impromptu 🗣️🗣️🗣️
{/* */}
{[...Array(50)].map((_, i) => { const size = 100 + Math.random() * 600; const rotation = Math.random() * 360; return ( ); })}

Speak Boldly

Transform your impromptu speaking skills with real-time insights and thoughtful prompts.

Our Features

Gemini-Powered Suggestions

Gemini-Powered Suggestions

Receive intelligent suggestions and insights from our Gemini AI, based on your speech notes, to enhance your impromptu speaking performance and content delivery.

Dynamic Topic Generator

Dynamic Topic Generator

Challenge yourself with our intelligent topic generator, designed to provide thought- [truncated — 669 more characters] ``` ### tailwind.config.js ```javascript /** @type {import('tailwindcss').Config} */ module.exports = { content: [ "./pages/**/*.{js,ts,jsx,tsx,mdx}", "./components/**/*.{js,ts,jsx,tsx,mdx}", "./app/**/*.{js,ts,jsx,tsx,mdx}", ], theme: { extend: { colors: { background: "var(--background)", foreground: "var(--foreground)", }, fontFamily: { inter: ['Inter', 'sans-serif'], }, }, }, plugins: [], }; ``` ### utils/cryptography.js ```javascript import CryptoJS from 'crypto-js'; const decrypt = (encryptedText, secretKey, fixedIV) => { const key = CryptoJS.enc.Utf8.parse(secretKey.padEnd(16, ' ')); // Pad to 16 bytes const iv = CryptoJS.enc.Utf8.parse(fixedIV.padEnd(16, ' ')); // Pad to 16 bytes const decrypted = CryptoJS.AES.decrypt(encryptedText, key, { iv: iv, mode: CryptoJS.mode.CBC, padding: CryptoJS.pad.Pkcs7, }); return decrypted.toString(CryptoJS.enc.Utf8); // Convert to UTF-8 string }; export { decrypt }; ``` ### utils/noteSuggestion.js ```javascript import { GoogleGenerativeAI } from '@google/generative-ai'; import { getNotesSuggestionsPrompt } from './prompt'; import dotenv from 'dotenv'; dotenv.config(); const genAI = new GoogleGenerativeAI(process.env.NEXT_PUBLIC_GEMINI_API_KEY); const model = genAI.getGenerativeModel({ model: 'gemini-1.5-flash', generationConfig: { temperature: 2, }, }); export const getSuggestionsFromGemini = async (topic, notes) => { try { const prompt = getNotesSuggestionsPrompt(topic, notes); const result = await model.generateContent(prompt); return result.response.text().trim(); } catch (error) { console.error('Error generating suggestion:', error); return ''; } }; ``` [19 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]