# Project export: Echo

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: TreeHacks 2025
- Tagline: Turn any PDF into an audiobook you can talk to
- Devpost: https://devpost.com/software/echo-q58c7p
- GitHub: https://github.com/685Degrees/echo-reader
- Video: https://www.youtube.com/embed/QrmbbdrFcOs?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent
- Team: 2 GitHub contributor(s) — henryweng03 (52 commits), Selena (6 commits)

## Devpost submission (written by the team)

### Inspiration

Over half of all readers listen to the audiobooks. Yet, only 9% of books are available as audiobooks - and that's doesn't count other written content, like articles, research papers, or blog posts.

### What it does

Echo takes any epub or PDF, cleans up any artifacts like footers/headers/page numbers, and turns it into an high quality audiobook within minutes - so you can listen to the latest research paper on your commute. Beyond this, you can TALK to Echo audiobooks. If you have a question about some confusing content, want to dive deeper into a concept, or even just define a word, you can ask Echo, which will respond with context from what you've been listening to :) How we made it We use React and PDF libraries to extract text from PDFs and EPUBs Because PDF has a more flexible format, the text extracted from the PDF is often messy, so we created a text cleaner with Google Gemini 2.0 that removes unwanted elements, like: Page numbers Headers and footers Copyright notices Formatting artifacts Books can be very large, so the text cleaner breaks the book up into many 1000 word chunks and cleans them in parallel in order to reduce latency. After we get the fully cleaned text, we convert it to audio using ElevenLabs. We stream the audio back to the user in chunks so they can start listening right away (as opposed to when everything is done generating. For the interactive audiobook feature, we built a context-aware system that tracks what users are currently listening to. This powers Echo's interactive features, allowing users to ask questions about the content and receive relevant responses based on their current position in the audiobook.

## README (from the GitHub repository)

# echo-reader


## Detected evidence (automated analysis)

Indexed codebase: 24 recognized source files, 77 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- Google Gemini (technology) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- Mistral AI (technology) — claimed on Devpost, not found in the code
- OpenAI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (34 of 34)

```
.DS_Store
echo-backend/.env.example
echo-backend/.gitignore
echo-backend/package.json
echo-backend/src/server.ts
echo-backend/tsconfig.json
echo-frontend/.gitignore
echo-frontend/eslint.config.mjs
echo-frontend/next.config.ts
echo-frontend/package.json
echo-frontend/postcss.config.mjs
echo-frontend/README.md
echo-frontend/src/components/AudioControls.tsx
echo-frontend/src/components/AudioProgress.tsx
echo-frontend/src/components/BookCard.tsx
echo-frontend/src/components/BookDropZone.tsx
echo-frontend/src/components/Header.tsx
echo-frontend/src/components/Typography.tsx
echo-frontend/src/hooks/useAudioPlayer.ts
echo-frontend/src/hooks/useWebRTC.ts
echo-frontend/src/lib/bookStorage.ts
echo-frontend/src/lib/utils.ts
echo-frontend/src/pages/_app.tsx
echo-frontend/src/pages/_document.tsx
echo-frontend/src/pages/api/save-audio.ts
echo-frontend/src/pages/api/tts-eleven.ts
echo-frontend/src/pages/book/[id].tsx
echo-frontend/src/pages/index.tsx
echo-frontend/src/styles/globals.css
echo-frontend/src/types/book.ts
echo-frontend/tailwind.config.js
echo-frontend/tailwind.config.ts
echo-frontend/tsconfig.json
README.md
```

### Dependencies

- echo-backend/package.json: @types/cors@^2.8.17, @types/express@^4.17.21, @types/node@^20.11.19, cors@^2.8.5, dotenv@^16.4.4, express@^4.18.2, ts-node-dev@^2.0.0, typescript@^5.3.3
- echo-frontend/package.json: @eslint/eslintrc@^3, @google/generative-ai@^0.21.0, @heroicons/react@^2.2.0, @prisma/client@^6.3.1, @types/formidable@^3.4.5, @types/node@^20, @types/react@^19, @types/react-dom@^19, @types/uuid@^10.0.0, clsx@^2.1.1, elevenlabs@^1.51.0, epubjs@^0.3.93, eslint@^9, eslint-config-next@15.1.7, formidable@^3.5.2, lucide-react@^0.475.0, next@15.1.7, postcss@^8, prisma@^6.3.1, react@^19.0.0, react-dom@^19.0.0, react-pdftotext@^1.3.4, slugify@^1.6.6, tailwind-merge@^3.0.1, tailwindcss@^3.4.1, typescript@^5, uuid@^11.0.5

### Recent commits (newest first)

- Merge pull request #4 from 685Degrees/henry/home-page-ui-referesh
- feat: save audio duration
- feat: update library on book save immediately
- feat: add library to home screen
- Merge pull request #3 from 685Degrees/henry/save-mp3
- feat: add edit book title
- feat: play audio from saved mp3 files
- feat: prevent saving until audio is fully generated
- feat: store audio as a mp3
- Merge pull request #2 from 685Degrees/henry/delete-book
- feat: add loading circle
- feat: get book by id instead of slug
- feat: add delete book functionality
- save in book drop zone doest allow u to save twice
- feat: fix useAudioPlayerHook so that it doesnt auto call audio gen
- feat: basic library saved stuff
- feat: create book context prompt
- feat: more audio progress bar fixing
- fix: audio progress bar
- sexify UI

## Key source files (fetched from GitHub, selected and truncated for size)

### echo-backend/package.json

```
{
  "name": "echo-backend",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "dev": "ts-node-dev --respawn --transpile-only src/server.ts",
    "build": "tsc",
    "start": "node dist/server.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "dependencies": {
    "@types/express": "^4.17.21",
    "@types/node": "^20.11.19",
    "cors": "^2.8.5",
    "@types/cors": "^2.8.17",
    "dotenv": "^16.4.4",
    "express": "^4.18.2",
    "ts-node-dev": "^2.0.0",
    "typescript": "^5.3.3"
  }
}

```

### echo-frontend/package.json

```
{
  "name": "echo-frontend",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev --turbopack",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@google/generative-ai": "^0.21.0",
    "@heroicons/react": "^2.2.0",
    "@prisma/client": "^6.3.1",
    "@types/formidable": "^3.4.5",
    "@types/uuid": "^10.0.0",
    "clsx": "^2.1.1",
    "elevenlabs": "^1.51.0",
    "epubjs": "^0.3.93",
    "formidable": "^3.5.2",
    "lucide-react": "^0.475.0",
    "next": "15.1.7",
    "prisma": "^6.3.1",
    "react": "^19.0.0",
    "react-dom": "^19.0.0",
    "react-pdftotext": "^1.3.4",
    "slugify": "^1.6.6",
    "tailwind-merge": "^3.0.1",
    "uuid": "^11.0.5"
  },
  "devDependencies": {
    "@eslint/eslintrc": "^3",
    "@types/node": "^20",
    "@types/react": "^19",
    "@types/react-dom": "^19",
    "eslint": "^9",
    "eslint-config-next": "15.1.7",
    "postcss": "^8",
    "tailwindcss": "^3.4.1",
    "typescript": "^5"
  }
}

```

### echo-backend/src/server.ts

```typescript
import express from "express";
import dotenv from "dotenv";
import cors from "cors";

dotenv.config();

const app = express();
const port = process.env.PORT || 3001;

app.use(cors());
app.use(express.json());

// Endpoint to create a new WebRTC session
app.post("/session", async (req, res) => {
  try {
    console.log("Creating new WebRTC session...");
    const requestBody = {
      model: "gpt-4o-realtime-preview-2024-12-17",
      voice: "alloy",
    };
    console.log("Request to OpenAI:", requestBody);

    const response = await fetch(
      "https://api.openai.com/v1/realtime/sessions",
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify(requestBody),
      }
    );

    if (!response.ok) {
      const errorText = await response.text();
      console.error("OpenAI API error:", {
        status: response.status,
        statusText: response.statusText,
        error: errorText,
      });
      throw new Error(`Failed to create session: ${errorText}`);
    }

    const data = await response.json();
    console.log("Session created successfully:", {
      sessionId: data.id,
      expiresAt: data.expires_at,
    });
    res.json(data);
  } catch (error) {
    console.error("Error creating session:", error);
    res.status(500).json({ error: "Failed to create session" });
  }
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

```

### echo-frontend/src/pages/index.tsx

```typescript
import { useState, useEffect } from "react";
import { BookDropZone } from "@/components/BookDropZone";
import { AudioProgress } from "@/components/AudioProgress";
import { AudioControls } from "@/components/AudioControls";
import { useWebRTC } from "@/hooks/useWebRTC";
import { useAudioPlayer } from "@/hooks/useAudioPlayer";
import { Header } from "@/components/Header";
import { Subheader } from "@/components/Typography";
import { BookCard } from "@/components/BookCard";
import { generateSpeech, saveAudioStream } from "@/lib/utils";
import { getBookMetadata, getBookById } from "@/lib/bookStorage";
import { Book } from "@/types/book";
import { v4 as uuidv4 } from "uuid";

export default function Home() {
  const [books, setBooks] = useState<Book[]>([]);
  const [bookText, setBookText] = useState("");
  const [isDiscussing, setIsDiscussing] = useState(false);
  const [saveStream, setSaveStream] = useState<ReadableStream | null>(null);

  const { isConnected, isConnecting, startSession, stopSession, error } =
    useWebRTC();

  const {
    isPlaying,
    isLoading,
    currentTimeSeconds,
    duration,
    bufferingProgress,
    handlePlayPause,
    handleSkipForward,
    handleSkipBack,
    handleProgressChange,
    setIsPlaying,
    setupAudioStream,
    isAudioReady,
  } = useAudioPlayer();

  useEffect(() => {
    loadBooks();
  }, []);

  const loadBooks = () => {
    const metadata = getBookMetadata();
    const fullBooks = metadata
      .map((meta) => {
        const book = getBookById(meta.id);
        return book;
      })
      .filter((book): book is Book => book !== null);
    setBooks(fullBooks);
  };

  const handleBookDelete = () => {
    loadBooks();
  };

  const handleBookUpdate = () => {
    loadBooks();
  };

  const handleTextExtracted = async (text: string) => {
    setBookText(text);
    try {
      const stream = await generateSpeech(text);
      // Clone the stream for potential future saving
      const [playStream, saveStream] = stream.tee();

      // Store the save stream for later use when saving
      setSaveStream(saveStream);

      // Setup audio for preview/playback
      await setupAudioStream(playStream);
    } catch (error) {
      console.error("Error generating speech:", error);
    }
  };

  // Modify handleSaveAudio to accept duration
  const handleSaveAudio = async (duration: number): Promise<string> => {
    if (!saveStream) {
      throw new Error("No audio stream available");
    }

    const bookId = uuidv4();
    const audioUrl = await saveAudioStream(saveStream, bookId);
    return audioUrl;
  };

  const handleDiscussToggle = async () => {
    if (isDiscussing) {
      setIsDiscussing(false);
      setIsPlaying(true);
      stopSession();
    } else {
      setIsDiscussing(true);
      // Pause audio when starting discussion
      if (isPlaying) {
        handlePlayPause();
      }
      if (!isConnected && !isConnecting) {
        await startSession();
      }
    }
  };

  const getButtonText = () => {
    if (isConnecting) return "Connecting...";
    if (isDiscussing) return "Resume listening";
    if (isPlaying) return "Pause & discuss";
    return "Discuss";
  };

  return (
    <div className="bg-primary-100 min-h-screen">
      <Header />
      <main className="pt-20 flex flex-col items-center p-8 space-y-12">
        {/* Library Section */}
        <div className="w-full max-w-2xl">
          <Subheader className="mb-6 text-primary-800">Your Library</Subheader>
          <div className="relative">
            <div className="overflow-x-auto pb-4 hide-scrollbar">
              <div className="flex space-x-6 w-max">
                {books.length === 0 ? (
                  <div className="flex items-center justify-center w-[320px] h-[200px] bg-primary-50 rounded-xl border border-primary-200 flex-shrink-0">
                    <p className="text-gray-500 text-center px-4">
                      No books in your library yet.
                      <br />
                      Upload a book below to get started.
                    </p>
                  </div>
                ) : (
                  [...books]
                    .reverse()
                    .map((book) => (
                      <BookCard
                        key={book.id}
                        id={book.id}
                        title={book.title}
                        lengthSeconds={book.lengthSeconds}
                        onDelete={handleBookDelete}
                        onUpdate={handleBookUpdate}
                      />
                    ))
                )}
              </div>
            </div>
            {/* Fade effect on the right */}
            <div className="absolute right-0 top-0 bottom-0 w-20 bg-gradient-to-l from-primary-100 to-transparent pointer-events-none" />
          </div>
        </div>

        {/* Upload Section */}
        <div className="w-full max-w-2xl">
          <Subheader className="mb-6 text-primary-800">Upload a book</Subheader>
          <BookDropZone
            onTextExtracted={handleTextExtracted}
            onSaveAudio={() => handleSaveAudio(duration || 0)}
            onBookSaved={loadBooks}
            text={bookText}
            duration={duration}
            isAudioReady={duration !== 0}
          />
        </div>

        {/* Audio Controls Section */}
        {bookText && (
          <div className="w-full max-w-2xl space-y-8 sm:space-y-12 flex flex-col items-center justify-center">
            <AudioProgress
              progressPercent={(currentTimeSeconds / (duration || 1)) * 100}
              bufferingProgress={bufferingProgress}
              duration={duration}
              currentTime={currentTimeSeconds}
              onChange={handleProgressChange}
            />
            <AudioControls
              isPlaying={isPlaying}
              isLoading={isLoading}
              onPlayPause={handlePlayPause}
              onSkipForward={handleSkipForward}
              onSkipBack={handleSkipB
[truncated — 570 more characters]
```

### echo-frontend/next.config.ts

```typescript
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  /* config options here */
  reactStrictMode: true,
};

export default nextConfig;

```

### echo-frontend/tailwind.config.ts

```typescript
import type { Config } from "tailwindcss";

export default {
  content: [
    "./src/pages/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/components/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/app/**/*.{js,ts,jsx,tsx,mdx}",
  ],
  theme: {
    extend: {
      colors: {
        background: "var(--background)",
        foreground: "var(--foreground)",
      },
    },
  },
  plugins: [],
} satisfies Config;

```

### echo-frontend/tailwind.config.js

```javascript
/** @type {import('tailwindcss').Config} */
module.exports = {
  content: [
    "./src/pages/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/components/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/app/**/*.{js,ts,jsx,tsx,mdx}",
  ],
  theme: {
    extend: {
      colors: {
        primary: {
          50: "#f4f4f2",
          100: "#e4e2dd",
          200: "#cac7be",
          300: "#aca698",
          400: "#948b7b",
          500: "#857b6d",
          600: "#72675c",
          700: "#5c534c",
          800: "#504843",
          900: "#453e3b",
          950: "#272321",
        },
      },
    },
  },
  plugins: [],
};

```

### echo-frontend/src/pages/_app.tsx

```typescript
import "@/styles/globals.css";
import type { AppProps } from "next/app";

export default function App({ Component, pageProps }: AppProps) {
  return <Component {...pageProps} />;
}

```

### echo-frontend/src/pages/_document.tsx

```typescript
import { Html, Head, Main, NextScript } from "next/document";

export default function Document() {
  return (
    <Html lang="en">
      <Head />
      <body className="antialiased">
        <Main />
        <NextScript />
      </body>
    </Html>
  );
}

```

### echo-frontend/src/types/book.ts

```typescript
export interface Book {
  id: string; // UUID
  bookSlug: string;
  title: string;
  text: string;
  audioUrl: string;
  lengthSeconds: number;
  createdAt: string;
}

export interface BookMetadata {
  id: string; // UUID
  bookSlug: string;
  title: string;
  lengthSeconds: number;
  createdAt: string;
}

```

[14 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]