# Project export: Recappd This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Tailoring the world's news into your perfect daily podcast real-time with AI. - Devpost: https://devpost.com/software/recappd-6ocn1z - GitHub: https://github.com/ibttf/recappd-ai - Demo: https://recappd-ai.vercel.app/ - Video: https://www.youtube.com/embed/WWvKcsBsNW0?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 1 GitHub contributor(s) — roylee0912 (19 commits) ## Devpost submission (written by the team) ### Inspiration Deployed on Vercel **See the code One of our teammates Cameron White is famous at our college, both for being busy, and for being a bit of a klutz. He's Grandmaster in League of Legends, has over 50k followers on Twitch, and is VERY often distracted by the latest in tech. Although we all admire how passionately he lives his life, this passion does not come without cost. Cameron does not know who the governor of California is and was very recently shocked to learn that Leonardo Da Vinci had already died. Poor Cameron! With the rise of the internet, faster communication, and long-distance transportation, you would think that people would know more about each other than ever, but that's just not true. Just like our buddy Cameron, people all around the world don't know what's NEW (get it, like the NEWs??). Recappd was designed to help people who struggle to find the time to keep up with the news. Real-time and powered by AI, Recappd allows users to create custom podcasts on the latest news that interests them, as frequently as they'd like, and as long as they'd like. ### What it does Recappd was designed to be intuitive. Just authenticate with Clerk using your Google or Github account, select a category of news that interests you (finance, tech, politics, etc), decide how long you want your podcasts to be, and how often you'd like to receive them, and that's it. Recappd will take your interests, query the world wide web for the latest news in these categories using dynamic REST API endpoints, utilize LLMs to accurately identify the most important and consistent points from the aggregate of these articles, and spin up a report in the form of an mp3 audiofile, saved in real-time on our Convex database. We've connected with cron workers via Convex to asynchronously schedule tasks and update your databases at regularly scheduled interviews, and allowed for full CRUD capabilities with Convex. ### How we built it We used NextJS 13's app router for the frontend with Typescript. For the styling, we used TailwindCSS, shadCN, and HeadlessUI. For authentication, we used Clerk, although it was largelyintegrated with Convex already. For the backend, we used Convex for our real-time database, including file storage and asynchronous, scheduled tasks via cron jobs. For the speech-to-text functionality, we used Google Cloud Platform. For the AI component, we used Open AI's GPT 3.5 model via their API. To get popular news sites, we used Newsdata's public API. ### Challenges we ran into Technically, utilizing any of the technologies offered at this hackathon proved to be quite difficult considering that we were working with audio files. As it turns out, it is rather challenging to upload audio files into databases, and even more troublesome to try and process and parse them. It took us many an hour at the Convex booth to figure out how we could get the audio files stored in our database for use later, which was a core functionality of the project. At the end of the day, it didn't actually end up being a challenging bytes/bits issue that we initially thought -- just a mismatch between the specific types of audio files used by Google and Convex. I could not give a bigger shoutout to the Convex team if I tried. They saved this project many times over. Another challenge we ran into was the input and output limits for both GPT 3.5 and Google Cloud Platform's Speech to Text API. There is a very thin rope to walk between trying to ambitiously grab too much data and have way too much output to reasonably convert and store into an mp3 audiofile, and having way too little input, resulting in podcasts that are either missing information or much shorter than they needed to be. We solved this by utilizing our own chunking solution, which JUST barely managed to keep us under the limit for ChatGPT to give us an output that would fit within both our information and time constraints. Another massive technical challenge was trying to upload our mp3's onto Youtube. Ffmpeg, the most popular library for converting mp3s to mp4s, was severely lacking in configuration information for Next JS and Typescript, and many other options online weren't APIs at all and required you manually insert them. No doubt this was a result of the difficulty of storing audio files in databases and the relative lack of utility for doing so in most other cases. Finally, likely the biggest challenge was not even a technical one. Wifi at this hackathon was extremely slow -- to the point that Cameron and Roy had to pivot out a few hours in due to their models not being able to train themselves with the limited internet capacities at the venue. ### Accomplishments we're proud of Undeniably the most impressive part about the project is our integration with Google Cloud Platform's Speech to Text API and Convex's real-time database. It took almost six hours of us swaying between believing it was possible and impossible, and it took another few hours for us to actually get to the concrete implementation. Thankfully, as a result, we can have audiofiles that are generated AND read by the client faster than it would have taken to read them outloud -- thanks to Convex's real-time database. We're also quite proud of our frontend and the styling. Though we didn't quite have the time to build out a fully responsive frontend, the styling was quite the challenge, particularly given that none of us had any prior expertise in this sort of UI/UX design. The design went through a few iterations to get to the finished product, and we really did finish right in the nick of time. I am also quite proud of our custom chunking system to allow ChatGPT to process these rather massive bits of data. By figuring out EXACTLY where our limit was in terms of GPT's inputs and carefully staying right below it, we're able to get the most accurate results possible from the model, with the least amount of hallucinations. ### What we learned As this was most of our members' first hackathon, it was really a great learning experience. It was very interesting to see that hackathons weren't as competitive as I made them out to be in my head and that it was oftentimes just a fun, collaborative place to be. It was also a truly eye-opening experience into the startup world, what with co-founders and founding engineers representing all the impressive booths and sponsors. It was a great chance to get to talk to people who were living my dream. ### What's next The one feature that I am quite regretful we didn't get to was integrating with an external API to host our audio files on. We started out hoping Spotify would have something, but they don't have a public API for something like that. As our choices grew smaller and smaller and our deadline continued to draw near, we decided to leave that as first up on our to-do list for the future. ## README (from the GitHub repository) ## Recappd AI ### General This project was built in 48 hours in a team of three as part of CalHacks 10.0. Deployed on: [Recappd](https://recappd.live) View Demo: [Demo](https://youtu.be/WWvKcsBsNW0) If the live doesn't work, then it very likely means we ran out of free credits for one of our various APIs. Let us know with a message or check out how it works in development by following the instructions below: ### Features - Authenticate using Clerk with Google or Github accounts. - Select your news category (finance, tech, politics, etc.). - Customize the podcast duration and frequency. - Recappd aggregates news using dynamic REST API endpoints. - Utilizes LLMs for content accuracy. - Generates an MP3 audio file stored in the Convex database. - Asynchronous updates using cron workers via Convex. - Full CRUD capabilities with Convex. - Technology Stack - Frontend: NextJS 13 with Typescript, TailwindCSS, shadCN, and HeadlessUI. - Authentication: Clerk (also integrated with Convex). - Backend: Convex for real-time data, file storage, and asynchronous cron tasks. - Speech-to-text: Google Cloud Platform. - AI: OpenAI's GPT 3.5 model. - News Retrieval: Newsdata's public API. - Deployment: Vercel - Recappd is a monorepo deployed at recapd.live. ### Development Setup To set up the project for development: ### Clone the repository. Create a .env file in the project root. Add the following environment variables to the .env file: ``` .env NEXT_PUBLIC_NEWS_API_KEY= NEXT_PUBLIC_OPENAI_API_KEY= NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL=/dashboard NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL=/dashboard NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY= CLERK_SECRET_KEY= GCP_PROJECT_ID= GCP_KEY_FILE_NAME= FREE_CONVERT_API_KEY= ``` Replace , , , , and with your actual values. You can copy and paste this into your README.md file. Make sure to replace the placeholder values with your actual keys when setting up the project. ## Detected evidence (automated analysis) Indexed codebase: 49 recognized source files, 82 KB. - CSS (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - OpenAI (technology) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code - TypeScript (language) — detected in the code ## Codebase structure (from repository index) ### Files (57 of 57) ``` .eslintrc.json .gitignore app/(auth)/sign-in/[[...sign-in]]/page.tsx app/(auth)/sign-up/[[...sign-up]]/page.tsx app/(landing)/layout.tsx app/(landing)/page.tsx app/api/gpt/route.ts app/api/recap/route.ts app/ConvexClientProvider.tsx app/dashboard/layout.tsx app/dashboard/page.tsx app/dashboard/recappd/page.tsx app/dashboard/settings/page.tsx app/dashboard/useStoreUserEffect.ts app/globals.css app/layout.tsx bun.lockb components.json components/dashboard-header.tsx components/dashboard-play.tsx components/dashboard-templates.tsx components/landing-hero.tsx components/landing-navbar.tsx components/Loading.tsx components/mobile-sidebar.tsx components/navbar.tsx components/sidebar.tsx components/ui/button.tsx components/ui/card.tsx components/ui/sheet.tsx components/ui/slider.tsx components/useCreateTemplateEffect.ts components/useDeleteRecapp.ts components/useDeleteTemplate.ts config.json convex/_generated/api.d.ts convex/_generated/api.js convex/_generated/dataModel.d.ts convex/_generated/server.d.ts convex/_generated/server.js convex/auth.config.js convex/crons.ts convex/http.ts convex/README.md convex/recapp.ts convex/schema.ts convex/templates.ts convex/tsconfig.json convex/users.ts lib/utils.ts middleware.ts next.config.js package.json postcss.config.js README.md tailwind.config.ts tsconfig.json ``` ### Dependencies - package.json: @clerk/clerk-react@^4.27.1, @clerk/nextjs@^4.26.1, @google-cloud/text-to-speech@^5.0.1, @headlessui/react@^1.7.17, @radix-ui/react-dialog@^1.0.5, @radix-ui/react-slider@^1.1.2, @radix-ui/react-slot@^1.0.2, @types/fluent-ffmpeg@^2.1.23, @types/node@^20, @types/react@^18, @types/react-dom@^18, autoprefixer@^10, class-variance-authority@^0.7.0, clsx@^2.0.0, convex@^1.5.1, dotenv@^16.3.1, eslint@^8, eslint-config-next@14.0.0, ffmpeg@^0.0.4, fluent-ffmpeg@^2.1.2, fs@^0.0.1-security, lucide-react@^0.290.0, next@14.0.0, openai@^4.14.1, postcss@^8, react@^18, react-dom@^18, react-icons@^4.11.0, tailwind-merge@^1.14.0, tailwindcss@^3, tailwindcss-animate@^1.0.7, typescript@^5, typewriter-effect@^2.21.0 ### Recent commits (newest first) - Update README.md - Update README.md - udpated favicon - Update README.md - Update README.md - separated response into two post requests so that my vercel woudln't time me out and break prod - split post request into two so that vercel timeout function woudl stop - shortened gpt delay to 1 second - added a delay in between gpt calls, think next server is getting overloaded - was trying to read a response body more than once - think everyrhing has to be strings for search params, hard coded it all in - p dangerous, added next public to env variable - unsure about urls earch params, added some weird parsing - typo. squahs later - added some other console log but not sure if this works, will squash this commit later - added some more testing, problem is with newsdata api in deploy - added error handling for when there are no recapps or templates found - added console logs for error handling in prodc - crons not working + update read me - crons not really working, restarted ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "recappd-ai", "version": "0.1.0", "private": true, "scripts": { "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint" }, "dependencies": { "@clerk/clerk-react": "^4.27.1", "@clerk/nextjs": "^4.26.1", "@google-cloud/text-to-speech": "^5.0.1", "@headlessui/react": "^1.7.17", "@radix-ui/react-dialog": "^1.0.5", "@radix-ui/react-slider": "^1.1.2", "@radix-ui/react-slot": "^1.0.2", "class-variance-authority": "^0.7.0", "clsx": "^2.0.0", "convex": "^1.5.1", "dotenv": "^16.3.1", "ffmpeg": "^0.0.4", "fluent-ffmpeg": "^2.1.2", "fs": "^0.0.1-security", "lucide-react": "^0.290.0", "next": "14.0.0", "openai": "^4.14.1", "react": "^18", "react-dom": "^18", "react-icons": "^4.11.0", "tailwind-merge": "^1.14.0", "tailwindcss-animate": "^1.0.7", "typewriter-effect": "^2.21.0" }, "devDependencies": { "@types/fluent-ffmpeg": "^2.1.23", "@types/node": "^20", "@types/react": "^18", "@types/react-dom": "^18", "autoprefixer": "^10", "eslint": "^8", "eslint-config-next": "14.0.0", "postcss": "^8", "tailwindcss": "^3", "typescript": "^5" } } ``` ### app/layout.tsx ```typescript import type { Metadata } from "next"; import "./globals.css"; import ConvexClientProvider from "./ConvexClientProvider"; export const metadata: Metadata = { title: "Recappd", description: "Your favorite news, recappd.", }; export default function RootLayout({ children, }: { children: React.ReactNode; }) { return ( {children} ); } ``` ### app/(landing)/page.tsx ```typescript import { LandingHero } from "@/components/landing-hero"; import { LandingNavbar } from "@/components/landing-navbar"; export default function Page() { return (
); } ``` ### app/(landing)/layout.tsx ```typescript const LandingLayout = ({ children }: { children: React.ReactNode }) => { return (
{children}
); }; export default LandingLayout; ``` ### app/dashboard/page.tsx ```typescript "use client"; import React from "react"; import DashboardHeader from "@/components/dashboard-header"; import DashboardTemplates from "@/components/dashboard-templates"; import useStoreUserEffect from "./useStoreUserEffect"; export default function Page() { const userId = useStoreUserEffect(); return (
); } ``` ### app/dashboard/layout.tsx ```typescript import Navbar from "@/components/navbar"; import Sidebar from "@/components/sidebar"; const DashboardLayout = ({ children }: { children: React.ReactNode }) => { return (
{children}
); }; export default DashboardLayout; ``` ### convex/_generated/server.js ```javascript /* eslint-disable */ /** * Generated utilities for implementing server-side Convex query and mutation functions. * * THIS CODE IS AUTOMATICALLY GENERATED. * * Generated by convex@1.5.1. * To regenerate, run `npx convex dev`. * @module */ import { actionGeneric, httpActionGeneric, queryGeneric, mutationGeneric, internalActionGeneric, internalMutationGeneric, internalQueryGeneric, } from "convex/server"; /** * Define a query in this Convex app's public API. * * This function will be allowed to read your Convex database and will be accessible from the client. * * @param func - The query function. It receives a {@link QueryCtx} as its first argument. * @returns The wrapped query. Include this as an `export` to name it and make it accessible. */ export const query = queryGeneric; /** * Define a query that is only accessible from other Convex functions (but not from the client). * * This function will be allowed to read from your Convex database. It will not be accessible from the client. * * @param func - The query function. It receives a {@link QueryCtx} as its first argument. * @returns The wrapped query. Include this as an `export` to name it and make it accessible. */ export const internalQuery = internalQueryGeneric; /** * Define a mutation in this Convex app's public API. * * This function will be allowed to modify your Convex database and will be accessible from the client. * * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. */ export const mutation = mutationGeneric; /** * Define a mutation that is only accessible from other Convex functions (but not from the client). * * This function will be allowed to modify your Convex database. It will not be accessible from the client. * * @param func - The mutation function. It receives a {@link MutationCtx} as its first argument. * @returns The wrapped mutation. Include this as an `export` to name it and make it accessible. */ export const internalMutation = internalMutationGeneric; /** * Define an action in this Convex app's public API. * * An action is a function which can execute any JavaScript code, including non-deterministic * code and code with side-effects, like calling third-party services. * They can be run in Convex's JavaScript environment or in Node.js using the "use node" directive. * They can interact with the database indirectly by calling queries and mutations using the {@link ActionCtx}. * * @param func - The action. It receives an {@link ActionCtx} as its first argument. * @returns The wrapped action. Include this as an `export` to name it and make it accessible. */ export const action = actionGeneric; /** * Define an action that is only accessible from other Convex functions (but not from the client). * * @param func - The function. It receives an {@link ActionCtx} as its first argument. * @returns The wrapped function. Include this as an `export` to name it and make it accessible. */ export const internalAction = internalActionGeneric; /** * Define a Convex HTTP action. * * @param func - The function. It receives an {@link ActionCtx} as its first argument, and a `Request` object * as its second. * @returns The wrapped endpoint function. Route a URL path to this function in `convex/http.js`. */ export const httpAction = httpActionGeneric; ``` ### app/dashboard/settings/page.tsx ```typescript "use client"; import { UserProfile } from "@clerk/clerk-react"; export default function Settings() { return (
); } ``` ### app/dashboard/recappd/page.tsx ```typescript "use client"; import DashboardTemplates from "@/components/dashboard-templates"; import useStoreUserEffect from "../useStoreUserEffect"; const NewsPage = () => { const userId = useStoreUserEffect(); return (

My Past Recapps

See the recapps you’ve made in the past.

); }; export default NewsPage; ``` ### app/api/recap/route.ts ```typescript import { NextResponse } from "next/server"; export async function POST(req: Request) { console.log("beginning of function"); //body doesn't have name and templateId anymore. //body returns articles data from newsdata api const baseUrl = "https://newsdata.io/api/1/news"; const body = await req.json(); const { category, podcastLength } = body; console.log(process.env.NEXT_PUBLIC_NEWS_API_KEY); const params = new URLSearchParams({ apikey: process.env.NEXT_PUBLIC_NEWS_API_KEY as string, timeframe: "48", category: category as string, full_content: "1", size: podcastLength as string, language: "en", }); console.log("params is good"); try { console.log(params); const respo = await fetch(`${baseUrl}?${params.toString()}`); if (!respo.ok) { throw new Error( `Network response from newsdata api was not ok. Response body:` ); } const data = await respo.json(); console.log( "network response from newsdata was okay and came back as a json." ); return NextResponse.json(data); } catch (error) { console.log("[CODE_ERROR]", error); return new NextResponse("Failed to fetch news", { status: 500 }); } } ``` [38 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]