# Project export: LiveStory This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 11.0 - Tagline: Talk to the characters in your favourite story! - Devpost: https://devpost.com/software/livestory - GitHub: https://github.com/pranavbedi/CalHacks2024 - Video: https://www.youtube.com/embed/E7Z3YmKI0LI?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (Vapi: Show us your Voice AI) - Team: 3 GitHub contributor(s) — Pranav Bedi (16 commits), connortbot (13 commits), 18gen (4 commits) ## Devpost submission (written by the team) ### Inspiration Our inspiration comes from one of our team member's bedtime story sessions with his little cousin. Whenever he reads him a picture book, his cousin always has questions like, "Why did Goldilocks walk into the house?"—and most of the time, we don’t have the answers. We wished there was a way for him to ask the characters directly, to hear their side of the story. That’s where the idea for LiveStory came from. We wanted to bring storybook characters to life and let kids interact with them directly to get answers to their questions in real time. ### What it does LiveStory is a children's storybook that comes alive! At ANY point in the story, interact with a character on the page and chat with them; how are they feeling about what just happened? With industry-leading AI voice-to-voice pipelines, readers can feel the emotion of their favourite characters. ### How we built it Our web application was built primarily using Reflex: Reflex for both frontend and backend React.js for managing the database of each page and character Custom assistants powered by Vapi, Groq, Deepgram, and 11Labs to simulate character interactions Reflex for Deploy ### Challenges we ran into We initially struggled with Reflex, but over time, it became our go-to tool for building the project. We had to prevent characters from spoiling the story by restricting their responses to what the reader had already seen. To solve this, we fed the accumulative story log into the voice API, ensuring characters only referenced the relevant parts of the story. ### Accomplishments we're proud of Completing the project on time and getting it fully functional!!! Learning Reflex and Lottie from scratch and successfully implementing them over the weekend. Collaborating with amazing Reflex engineers to create a solid product based on their platform. Committing 20+ hours and $1,000 on travel from Waterloo, Canada, to make this hackathon happen! ### What we learned Making a full-stack app in Reflex Implementing beautiful vector animations with Lottie Implementing voice-to-voice models in web apps ### What's next As time is the biggest limitation during a hackathon, we would have loved to pour more time into the art to make a more beautiful experience. More stories! More animations! Characters could, based on the emotions of their speech, have reactive animations Character Interaction Interface: a more advanced UI can note your emotions Choose your own adventures! With supported stories, the conversations with the characters could influence the story! Customization for reader! We can also try feeding reader's information such as name, hobby and academic interest to serve better user experience. ## README (from the GitHub repository) # LiveStory - CalHacks 11.0 Winner! ## Video Demo [![Video Demo Of Live Story](https://img.youtube.com/vi/E7Z3YmKI0LI/0.jpg)](https://www.youtube.com/watch?v=E7Z3YmKI0LI) ## Inspiration Our inspiration comes from one of our team member's bedtime story sessions with his little cousin. Whenever he reads him a picture book, his cousin always has questions like, "Why did Goldilocks walk into the house?"—and most of the time, we don’t have the answers. We wished there was a way for him to ask the characters directly, to hear their side of the story. That’s where the idea for LiveStory came from. We wanted to bring storybook characters to life and let kids interact with them directly to get answers to their questions in real time. ## What it does **LiveStory** is a children's storybook that comes alive! At ANY point in the story, interact with a character on the page and chat with them; how are they feeling about what just happened? With industry-leading AI voice-to-voice pipelines, readers can feel the emotion of their favourite characters. ## How we built it Our web application was built primarily using Reflex: - Reflex for both frontend and backend - React.js for managing the database of each page and character - Custom assistants powered by Vapi, Groq, Deepgram, and 11Labs to simulate character interactions - Reflex for Deployment ## Challenges we ran into - We initially struggled with Reflex, but over time, it became our go-to tool for building the project. - We had to prevent characters from spoiling the story by restricting their responses to what the reader had already seen. To solve this, we fed the accumulative story log into the voice API, ensuring characters only referenced the relevant parts of the story. ## Accomplishments that we're proud of - Completing the project on time and getting it fully functional!!! - Learning Reflex and Lottie from scratch and successfully implementing them over the weekend. - Collaborating with amazing Reflex engineers to create a solid product based on their platform. - Committing 20+ hours and $1,000 on travel from Waterloo, Canada, to make this hackathon happen! ## What we learned - Making a full-stack app in Reflex - Implementing beautiful vector animations with Lottie - Implementing voice-to-voice models in web apps ## What's next for LiveStory As time is the biggest limitation during a hackathon, we would have loved to pour more time into the art to make a more beautiful experience. - More stories! - More animations! Characters could, based on the emotions of their speech, have reactive animations - Character Interaction Interface: a more advanced UI can note your emotions - Choose your own adventures! With supported stories, the conversations with the characters could influence the story! - Customization for reader! We can also try feeding reader's information such as name, hobby and academic interest to serve better user experience. # Screenshots ![image](https://github.com/user-attachments/assets/7490cda9-6bf6-4014-a827-05d235bc6c36) ![image](https://github.com/user-attachments/assets/1db5c46d-37ce-4b71-818e-8d2da4d7a3f3) **Built by Connor Loi, Gen Ichihashi and Pranav Bedi** ## Detected evidence (automated analysis) Indexed codebase: 34 recognized source files, 93 KB. - CSS (language) — detected in the code - JavaScript (language) — detected in the code - Next.js (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Tailwind CSS (technology) — detected in the code ## Codebase structure (from repository index) ### Files (63 of 63) ``` .gitignore .web/.gitignore .web/bunfig.toml .web/components/reflex/radix_themes_color_mode_provider.js .web/env.json .web/jsconfig.json .web/next.config.js .web/package.json .web/pages/_app.js .web/pages/_document.js .web/pages/404.js .web/pages/index.js .web/postcss.config.js .web/reflex.install_frontend_packages.cached .web/reflex.json .web/styles/styles.css .web/styles/tailwind.css .web/tailwind.config.js .web/utils/client_side_routing.js .web/utils/components.js .web/utils/context.js .web/utils/helpers/dataeditor.js .web/utils/helpers/debounce.js .web/utils/helpers/paste.js .web/utils/helpers/range.js .web/utils/helpers/throttle.js .web/utils/state.js .web/utils/stateful_components.js .web/utils/theme.js app/.gitignore app/app/__init__.py app/app/app.py app/app/characterComponent.py app/app/getPages.py app/app/navBar.py app/app/pages/story_page.py app/assets/CharacterComponent.js app/assets/lottie-animations/first-pig-talking.json app/assets/lottie-animations/first-pig-thinking.json app/assets/lottie-animations/pig-wheel.json app/assets/lottie-animations/second-pig-talking.json app/assets/lottie-animations/second-pig-thinking.json app/assets/lottie-animations/third-pig-talking.json app/assets/lottie-animations/third-pig-thinking.json app/assets/lottie-animations/wolf-talking.json app/assets/lottie-animations/wolf-thinking.json app/requirements.txt app/rxconfig.py frontend/characterComponent.js frontend/lottie-animations/first-pig-talking.json frontend/lottie-animations/first-pig-thinking.json frontend/lottie-animations/pig-wheel.json frontend/lottie-animations/second-pig-talking.json frontend/lottie-animations/second-pig-thinking.json frontend/lottie-animations/third-pig-talking.json frontend/lottie-animations/third-pig-thinking.json frontend/lottie-animations/wolf-talking.json frontend/lottie-animations/wolf-thinking.json frontend/lottie.js frontend/VapiCallButton.js README.md requirements.txt rxconfig.py ``` ### Dependencies - .web/package.json: @babel/standalone@7.25.7, @emotion/react@11.13.3, @radix-ui/themes@^3.0.0, @tailwindcss/typography@^0.5.15, autoprefixer@10.4.20, axios@1.7.7, json5@2.2.3, lottie-react@^2.4.0, lottie-web@^5.12.2, lucide-react@^0.359.0, next@14.2.14, next-sitemap@4.2.3, next-themes@0.3.0, postcss@8.4.47, postcss-import@16.1.0, react@18.3.1, react-dom@18.3.1, react-focus-lock@2.13.2, socket.io-client@4.8.0, sonner@^1.5.0, tailwindcss@^3.4.13, universal-cookie@7.2.0 - app/requirements.txt: reflex@==0.6.3 - requirements.txt: hume@==0.4.2, reflex@==0.5.10 ### Recent commits (newest first) - Update README.md - Update README.md - Update README.md - Update README.md - Update README.md - responsive - talking comes to front - env for vapi key - better interaction animation - instant switch to lottie - pycache - uploading .web folder (used to be ignored) - Update .gitignore - Update characterComponent.py - Data retrival from db + story population - Fixed Gen's issue with data retrirval + added stories - old background - config file - reqs for deployment - Update .gitignore ## Key source files (fetched from GitHub, selected and truncated for size) ### requirements.txt ``` hume==0.4.2 reflex==0.5.10 ``` ### app/requirements.txt ``` reflex==0.6.3 ``` ### .web/package.json ``` { "name": "reflex", "scripts": { "dev": "next dev", "export": "next build", "export-sitemap": "next build && next-sitemap", "prod": "next start" }, "dependencies": { "@babel/standalone": "7.25.7", "@emotion/react": "11.13.3", "@radix-ui/themes": "^3.0.0", "@tailwindcss/typography": "^0.5.15", "axios": "1.7.7", "json5": "2.2.3", "lottie-react": "^2.4.0", "lottie-web": "^5.12.2", "lucide-react": "^0.359.0", "next": "14.2.14", "next-sitemap": "4.2.3", "next-themes": "0.3.0", "react": "18.3.1", "react-dom": "18.3.1", "react-focus-lock": "2.13.2", "socket.io-client": "4.8.0", "sonner": "^1.5.0", "tailwindcss": "^3.4.13", "universal-cookie": "7.2.0" }, "devDependencies": { "autoprefixer": "10.4.20", "postcss": "8.4.47", "postcss-import": "16.1.0" } } ``` ### app/app/app.py ```python import reflex as rx from rxconfig import config from .pages.story_page import story from .getPages import get_page from .navBar import navbar_searchbar from .characterComponent import InteractableCharacter class State(rx.State): talking: bool = False def start_talking(self): self.talking = True def stop_talking(self): self.talking = False def index() -> rx.Component: # Landing page return rx.vstack( navbar_searchbar(), rx.hstack( rx.link( rx.image( src="ThreeLittlePigs.jpg", width="auto", height="200px", margin="5px", border = " 3px solid green", border_radius="10px 50px 50px 10px", ), href="/story", ), rx.link( # Place the block before the width argument rx.image( src="WeAreAll.png", width="auto", height="200px", border = " 3px solid brown", margin="5px", border_radius="10px 50px 50px 10px", ), href="/404", ), rx.link( # Place the block before the width argument rx.image( src="muddle.jpg", width="auto", height="200px", border = " 3px solid purple", margin="5px", border_radius="10px 50px 50px 10px", ), href="/404", ), ), rx.hstack( rx.link( rx.image( src="caterpillar.png", width="auto", height="200px", margin="5px", border = " 3px solid red", border_radius="10px 50px 50px 10px", ), href="/404", ), rx.link( # Place the block before the width argument rx.image( src="ribbit.jpg", width="auto", height="200px", margin="5px", border = " 3px solid green", border_radius="10px 50px 50px 10px", ), href="/404", ), rx.link( # Place the block before the width argument rx.image( src="bear.jpeg", width="auto", height="200px", margin="5px", border = " 3px solid yellow", border_radius="10px 50px 50px 10px", ), href="/404", ), ), spacing="5", align="center", justify="center", height="100vh", bg="beige", ) app = rx.App() app.add_page(index) app.add_page(story, route="/story") app.api.add_api_route("/getPages/{pageIndex}", get_page) ``` ### .web/pages/index.js ```javascript /** @jsxImportSource @emotion/react */ import { Fragment, useCallback, useContext, useEffect, useState } from "react" import { ColorModeContext, EventLoopContext } from "/utils/context" import { Event, getBackendURL, isTrue, refs } from "/utils/state" import { MoonIcon as LucideMoonIcon, SunIcon as LucideSunIcon, WifiOffIcon as LucideWifiOffIcon } from "lucide-react" import { keyframes } from "@emotion/react" import { toast, Toaster } from "sonner" import env from "/env.json" import { Button as RadixThemesButton, Code as RadixThemesCode, Container as RadixThemesContainer, Flex as RadixThemesFlex, Heading as RadixThemesHeading, IconButton as RadixThemesIconButton, Link as RadixThemesLink, Text as RadixThemesText } from "@radix-ui/themes" import NextLink from "next/link" import NextHead from "next/head" export function Toaster_9d6e054b03c6e5d1bea1c0a5576b4e6d () { const { resolvedColorMode } = useContext(ColorModeContext) refs['__toast'] = toast const [addEvents, connectErrors] = useContext(EventLoopContext); const toast_props = ({ ["description"] : ("Check if server is reachable at "+getBackendURL(env.EVENT).href), ["closeButton"] : true, ["duration"] : 120000, ["id"] : "websocket-error" }); const [userDismissed, setUserDismissed] = useState(false); (useEffect( () => { if ((connectErrors.length >= 2)) { if (!userDismissed) { toast.error( `Cannot connect to server: ${((connectErrors.length > 0) ? connectErrors[connectErrors.length - 1].message : '')}.`, {...toast_props, onDismiss: () => setUserDismissed(true)}, ) } } else { toast.dismiss("websocket-error"); setUserDismissed(false); // after reconnection reset dismissed state } } , [connectErrors])) return ( ) } export function Link_af00c102e2c2a7018dd100f1a1f1b98d () { return ( {"Check out our docs!"} ) } const pulse = keyframes` 0% { opacity: 0; } 100% { opacity: 1; } ` export function Iconbutton_52395c54b68ff5a6f498d43b99daaac3 () { const { toggleColorMode } = useContext(ColorModeContext) const [addEvents, connectErrors] = useContext(EventLoopContext); const on_click_9922dd3e837b9e087c86a2522c2c93f8 = useCallback(toggleColorMode, [addEvents, Event, toggleColorMode]) return ( ) } export function Fragment_e521b13e556da291bcec5187a783ea81 () { const [addEvents, connectErrors] = useContext(EventLoopContext); return ( {isTrue((connectErrors.length > 0)) ? ( ) : ( )} ) } export function Div_24a2e81d0c5d3cb5b5f786fdef44e514 () { const [addEvents, connectErrors] = useContext(EventLoopContext); return (
0) ? connectErrors[connectErrors.length - 1].message : ''))}>
) } export function Fragment_eb4ad804ee463e78d3c73fc2a2903123 () { const { resolvedColorMode } = useContext(ColorModeContext) return ( {isTrue((resolvedColorMode === "light")) ? ( ) : ( )} ) } export function Link_3607b2ee649d0f223249459744b7335b () { const { resolvedColorMode } = useContext(ColorModeContext) return ( {"Built with "}