# Project export: StoryTime

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: A personalized digital storytelling experience designed to bring inclusivity and creativity to every child’s bedtime story.
- Devpost: https://devpost.com/software/storytime-zgml96
- GitHub: https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- Demo: https://www.canva.com/design/DAGfRF572EU/x1Ns1a9Y5e7Tk0JMgv5Hag/view?utm_content=DAGfRF572EU&utm_campaign=designshare&utm_medium=link2&utm_source=uniquelinks&utlId=h2c14011d0f
- Video: https://www.youtube.com/embed/rY1RI-wA46k?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — naishagarwal (49 commits), hypatiav2 (29 commits)

## Devpost submission (written by the team)

### Inspiration

One of the most universal experiences of childhood is storytelling—whether through books, TV shows, or cherished tales passed down by loved ones. Storytelling is at the heart of human existence, shaping young minds, imparting valuable lessons, and creating lasting memories. Beyond the wisdom these stories carry, early engagement with reading is one of the strongest predictors of future success. It's clear that storytelling plays a crucial role in a child's development—but what stories are available for them to read? Many traditional fairytales and popular children's books fail to truly reflect a child’s unique culture, identity, and personal experiences. Meanwhile, parents often struggle to create engaging, meaningful stories on the spot—ones that not only captivate their children but also instill a lifelong love of reading. Enter StoryTime—a personalized digital storytelling experience designed to bring inclusivity, creativity, and educational value to every child’s bedtime story.

### What it does

StoryTime generates custom, interactive storybooks based on a child’s profile and a lesson or theme chosen by the parent. It features AI-generated stories tailored to the child’s background and interests, and narrated audio to bring the stories to life. There are AI-generated animations for every page, making the experience more immersive. Our platform is also hands-free and accessible, with the capability to simply talk to the platform and watch your story unfold. StoryTime allows children to have their own personal library with engaging, educational books at their fingertips.

### How we built it

Our tech stack includes: FastAPI for backend development. OpenAI’s GPT models to generate personalized stories, and their text to speech API for narration OpenAI’s Whisper model for speech-to-text functionality. LumaAI to generate AI-powered video content. React & Figma for a seamless and visually engaging frontend experience.

### Challenges we ran into

Some challenges we ran into was video latency. Generating videos took around 15-20 seconds per page, which posed a challenge in maintaining a smooth reading experience. To counter this, we implemented background processing, ensuring videos generate progressively while the user reads or listens to the story. This way, the next video is ready just in time. Additionally, integrating OpenAI’s Whisper model with our frontend posed technical challenges. Handling different file formats and ensuring smooth real-time transcription required multiple iterations.

### Accomplishments we're proud of

We are proud of having a fully functional, end-to-end product that aligns with our original vision. There is seamless AI integration across storytelling, voice narration, and video generation. Most importantly, our platform makes personalized storytelling more engaging and representative of diverse cultures.

### What we learned

We learned about seamlessly integrating between the frontend and the backend, as well as being able to use Luma's product and prompt engineer to get as consistent of videos as possible. We also learned how to handle asynchronous generation efficiently.

### What's next

We want to work on having better video consistency and ensuring character and style continuity across pages. We would also like to experiment with extended animations and video lengths for richer storytelling, and give users more opportunities to customize their story, including animation styles, narration voices, and more. We would also like to add the capability of live interruption, where children can stop the flow of the story as it is generating and change the plot, characters, or anything else, and the story will accordingly adjust.

## README (from the GitHub repository)

A personalized digital storytelling experience designed to bring inclusivity, creativity, and educational value to every child’s bedtime story.

How to start:

frontend: npm start
backend: python main.py

Will need API keys for Luma and OpenAI



## Detected evidence (automated analysis)

Indexed codebase: 21 recognized source files, 45 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code

## Codebase structure (from repository index)

### Files (27 of 27)

```
.gitignore
backend/db.json
backend/generate.py
backend/main.py
backend/story-generation.py
backend/story.json
backend/tts.py
frontend/package.json
frontend/postcss.config.js
frontend/public/index.html
frontend/src/App.js
frontend/src/components/NavBar.js
frontend/src/components/Storybook.js
frontend/src/index.css
frontend/src/index.js
frontend/src/pages/Generate.js
frontend/src/pages/Home.css
frontend/src/pages/Home.js
frontend/src/pages/Profile.js
frontend/src/pages/Saved.css
frontend/src/pages/Saved.js
frontend/src/pages/scene.js
frontend/src/pages/StoryPage.js
frontend/tailwind.config.js
README.md
requirements.txt
todos.txt
```

### Dependencies

- frontend/package.json: @tailwindcss/forms@^0.5.3, @tailwindcss/postcss@^4.0.6, autoprefixer@^10.4.20, eslint@^8.57.0, eslint-plugin-react@^7.34.0, howler@^2.2.4, postcss@^8.5.2, react@^18.2.0, react-dom@^18.2.0, react-icons@^5.4.0, react-router-dom@^6.15.0, react-scripts@5.0.1, tailwindcss@^3.4.17, three@^0.173.0
- requirements.txt: google-genai, lumaai, openai, tinydb

### Recent commits (newest first)

- final description
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- minor fix
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- final touches
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- final touches
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- cleaning up
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- refactor and fix backend db code
- cleaned
- transcribe api
- removing background audio generation
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- add saved stories api
- revert
- change in create story
- Merge branch 'main' of https://github.com/naishagarwal/treehacks-2025-personalized-storybook
- add tts support

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
lumaai
google-genai
openai
tinydb
```

### frontend/package.json

```
{
    "name": "storybook-ai",
    "version": "1.0.0",
    "private": true,
    "scripts": {
        "start": "react-scripts start",
        "build": "react-scripts build",
        "test": "react-scripts test",
        "eject": "react-scripts eject"
    },
    "dependencies": {
        "@tailwindcss/forms": "^0.5.3",
        "@tailwindcss/postcss": "^4.0.6",
        "autoprefixer": "^10.4.20",
        "howler": "^2.2.4",
        "postcss": "^8.5.2",
        "react": "^18.2.0",
        "react-dom": "^18.2.0",
        "react-icons": "^5.4.0",
        "react-router-dom": "^6.15.0",
        "react-scripts": "5.0.1",
        "tailwindcss": "^3.4.17",
        "three": "^0.173.0"
    },
    "devDependencies": {
        "eslint": "^8.57.0",
        "eslint-plugin-react": "^7.34.0"
    },
    "browserslist": {
        "production": [
            ">0.2%",
            "not dead",
            "not op_mini all"
        ],
        "development": [
            "last 1 chrome version",
            "last 1 firefox version",
            "last 1 safari version"
        ]
    }
}

```

### backend/main.py

```python
from fastapi import FastAPI, HTTPException, BackgroundTasks, Body, Response, UploadFile, File
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Dict, List, Optional
#from google import genai
from openai import OpenAI
import json
import os
from tinydb import TinyDB, Query
import time
from dotenv import load_dotenv
from lumaai import LumaAI
import uuid
from pathlib import Path
import io

app = FastAPI()

# Configure CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:3000"],  # Adjust based on your frontend URL
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

load_dotenv()
# Load API keys from environment variables
LUMAAI_API_KEY = os.getenv("LUMAAI_API_KEY")
#GEMINI_API_KEY = os.getenv("GEMINI_API_KEY")
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

# Initialize clients
#genai_client = genai.Client(api_key=GEMINI_API_KEY)
luma_client = LumaAI(auth_token=LUMAAI_API_KEY)
openai_client = OpenAI(api_key=OPENAI_API_KEY)

# Directories for audio files (since they are being stored locally)
AUDIO_DIR = Path("tts_outputs")
AUDIO_DIR.mkdir(exist_ok=True)

# Database initialization and table creation
db = TinyDB('db.json')
videos_table = db.table("page_videos")
profile_table = db.table("profiles")
stories_table = db.table("stories")

class Profile(BaseModel):
    nickname: str
    age: int
    location: str
    gender: str
    race: str
    interests: str

class StoryRequest(BaseModel):
    user_input: str
    child_profile: Profile

def generate_story_prompt(user_input: str, child_profile: Profile) -> str:
    return f"""Generate a story with the following input: {user_input}. 
    This story is being told to {child_profile.nickname}, a {child_profile.age} year old {child_profile.gender} from {child_profile.location} who is {child_profile.race} and enjoys {child_profile.interests}. If any of those fields seem missing, ignore and 
    proceed by making inferences or make the story broadly applicable.
    Please provide an appropriate children's story given this information, and make it personalized for either the listener's background or interests when possible. This should not include any role-playing with you as the parent, just the 
    story. The characters in the story do not necessarily have to be the listener, they can be new characters or animals or creatures as well. Try to show more than tell, and use onomatopoeia occassionally where it makes sense.
    Additionally, please divide up the story into multiple pages, just like a regular children's book. Return the final output in a JSON format,
    where the keys are "story" and "pages", and the values are a short title for the story and the list of pages, respectively. DO NOT include Page 1, Page 2, etc in the text you return, just the actual content. Besides these elements,
    there should be no other additional output. I should be able to use the command json.loads(output) to get the story title and the list of pages. That means "pages" should simply map to a list of strings, with each string being the text for the page.
    Try to ensure the story has an overarching, interesting plot with a relevant story arc. Be subtle about the lessons taught, opting to have an engaging story over overexplaining. Use language appropriate for children's stories,
    with repetitive phrasing where applicable and some challenge words appropriate for their age.
    """
# Video Generation Helper Functions

def generate_character_physical_description(story):
  response = openai_client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "You are a helpful assistant generating children's stories."},
            {
                "role": "user",
                "content": "Please generate a physical description of the character in the following story. Include vivid description, enough to simulate the character in a video. Limit description to very concise direct physical attributes, including clothing. Here is the story: " + story
            }
        ]
    )

  return response.choices[0].message.content

def generate_video_for_page(physical_description: str, page: str, style: str) -> str:
    if isinstance(page, dict):
        page = page.get("content", str(page))

    starter_prompt = (
        f"Please generate a video for the following page, in a {style} style. "
        f"If the page involves the character, use this physical description: {physical_description}. "
        "If not, generate a video without the character. Here is the page: "
    )
    prompt = starter_prompt + page
    print("Video prompt:", prompt)
    
    generation = luma_client.generations.create(prompt=prompt)
    completed = False
    while not completed:
        generation = luma_client.generations.get(id=generation.id)
        if generation.state == "completed":
            completed = True
        elif generation.state == "failed":
            raise RuntimeError(f"Video generation failed: {generation.failure_reason}")
    
    video_url = generation.assets.video
    print("Generated video URL:", video_url)
    return video_url

def background_generate_video(story_id: int, description: str, page_number: str, page_content: str, style: str):
    """
    Background task to generate a video for a single page and update its status in TinyDB.
    """
    try:
        video_url = generate_video_for_page(description, page_content, style)
        Video = Query()
        record = videos_table.get(Video.story_id == story_id)
        if record:
            pages = record.get("pages", {})
            pages[page_number]["video_url"] = video_url
            videos_table.update({"pages": pages}, Video.story_id == story_id)
            print(f"Updated story {story_id}, page {page_number} with video URL.")
    except Exception as e:
        Video = Query()
        record = videos_table.get(Video.story_id == story_id)
        i
[truncated — 6347 more characters]
```

### frontend/src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';  // You can add your global styles here
import App from './App';  // Make sure this points to your main App component

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);
```

### frontend/src/App.js

```javascript
import { BrowserRouter as Router, Routes, Route } from "react-router-dom";
import Generate from "./pages/Generate";
import StoryPage from "./pages/StoryPage";
import Saved from "./pages/Saved";
import Profile from "./pages/Profile";
import Home from "./pages/Home";

function App() {
  return (
    <Router>
      <Routes>
        <Route path="/" element={<Home />} />
        <Route path="/generate" element={<Generate />} />
        <Route path="/story/:storyId" element={<StoryPage />} />
        <Route path="/saved" element={<Saved />} />
        <Route path="/profile" element={<Profile />} />
      </Routes>
    </Router>
  );
}

export default App;

```

### frontend/postcss.config.js

```javascript
// postcss.config.js
module.exports = {
    plugins: [
      require('tailwindcss'),   // TailwindCSS plugin
      require('autoprefixer'),   // Autoprefixer plugin
    ],
  }
```

### backend/tts.py

```python
from pathlib import Path
import openai

speech_file_path = Path(__file__).parent / "speech.mp3"
response = openai.audio.speech.create(
  model="tts-1",
  voice="ash",
  input="The quick brown fox jumped over the lazy dog."
)
response.stream_to_file(speech_file_path)
```

### frontend/tailwind.config.js

```javascript
// tailwind.config.js
module.exports = {
  content: [
    './src/**/*.{js,jsx,ts,tsx}',  // Update to ensure Tailwind scans your source files
  ],
  theme: {
    extend: {
      fontFamily: {
        serif: ['Merriweather', 'serif'],
      },
    }
  },
  plugins: [],
}

```

### backend/generate.py

```python
import openai

openai.api_key = "YOUR_OPENAI_API_KEY"

def generate_story(description):
    # Generate text
    story_text = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[{"role": "system", "content": f"Generate a children's story about: {description}"}]
    )["choices"][0]["message"]["content"]

    # Split into pages and generate images
    story_parts = story_text.split(". ")
    pages = []
    for part in story_parts:
        image = openai.Image.create(
            prompt=f"Illustration of: {part}",
            n=1,
            size="512x512"
        )["data"][0]["url"]
        pages.append({"text": part, "image": image})
    return pages

```

### backend/story-generation.py

```python
## Taking user prompt and generating the story
from google import genai
from openai import OpenAI
import json
import ast

GEMINI_API_KEY = "AIzaSyBAvl0X_PmmWJpWkHuz5NFZQQOuw42icPM"
OPENAI_API_KEY = "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"
client = genai.Client(api_key= GEMINI_API_KEY)
client2 = OpenAI(api_key = OPENAI_API_KEY)


user_input = "Tell my child a story about how patience is a virtue."
child_profile_info = {
    "name": "Emily",
    "age": 7,
    "gender": "female",
    "location": "New York",
    "race": "latina",
    "interests": "reading, playing with toys"
}

def generate_story_prompt(user_input, child_profile_info):
    name = child_profile_info["name"]
    age = child_profile_info["age"]
    gender = child_profile_info["gender"]
    location = child_profile_info["location"]
    race = child_profile_info["race"]
    interests = child_profile_info["interests"]
    
    prompt = f"""Generate a story with the following input from a parent: {user_input}. 
    They are telling this story to {name}, a {age} year old {gender} from {location} who is {race} and enjoys {interests}. 
    Please provide an appropriate children's story given this information, and make it personalized to {name}. This should not include any role-playing with you as the parent, just the 
    story.
    Additionally, please divide up the story into multiple pages, just like a regular children's book. Return the final output in a JSON format,
    where the keys are "story" and "pages", and the values are the story and the list of pages, respectively. DO NOT include Page 1, Page 2, etc in the story. Besides these elements,
    there should be no other additional output. I should be able to use the command json.loads(output) to get the story and list of pages.
    """
    
    return prompt

def generate_story():
    story_prompt = generate_story_prompt(user_input, child_profile_info)

    ## Using Google Gemini 
    # response = client.models.generate_content(
    #     model="gemini-2.0-flash",
    #     contents = {story_prompt}
    # )

    # return response.text

    ## Using OpenAI
    response = client2.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are a helpful assistant generating children's stories."},
            {
                "role": "user",
                "content": story_prompt
            }
        ]
    )

    return response.choices[0].message.content

story = generate_story() 
# Save story to a file
page_list = json.loads(story) # json dictionary of the story and the pages
# print(type(story))
# print(story)
# print(type(page_list))
# print(page_list)

# Save page_list into a json file
with open('story.json', 'w') as f:
    json.dump(page_list, f)


# generating story with specific page number
```

[12 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]