# Project export: How To ANYTHING

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 10.0
- Tagline: How To ANYTHING is your one-stop platform for learning any task or skill. Using generative AI and stable diffusion, we can break down any complex task into a simple one.
- Devpost: https://devpost.com/software/how-to-anything
- GitHub: https://github.com/JayYeung/calhacks2023
- Team: 2 GitHub contributor(s) — JayYeung (9 commits), Nathan Shum (2 commits)

## Devpost submission (written by the team)

### Inspiration

We recognized the overwhelming amount of disparate information scattered across the web. This made it hard for individuals to find concise, reliable instructions for a wide array of tasks. "How To ANYTHING" was born out of the desire to streamline this process, turning any query into a detailed guide.

### What it does

"How To ANYTHING" transforms user queries into step-by-step guides akin to WikiHow articles. By leveraging advanced text and image generation techniques, our platform creates uniquely tailored instructions with associated images, ensuring that even the most intricate tasks become straightforward.

### How we built it

The foundation of our platform lies in the integration of state-of-the-art text generation and image synthesis technologies. We utilize LLMs for generating detailed, accurate textual content, while sophisticated algorithms create corresponding images that enhance comprehension. We used Together AI, Aleo, zillisDB, and HUME AI The system is built on the Next.js / FastAPI framework, ensuring a seamless and responsive user experience.

### Challenges we ran into

Token Limitations: When we initially built our "How to Anything" tool, we faced significant challenges due to token limitations in our text-to-image model. Many complex tasks and detailed instructions required a large number of tokens to convey accurately. This limitation hampered our ability to generate comprehensive and detailed step-by-step instructions, which was a roadblock for providing valuable user experiences. Transition to Together AI: To address the token limitations and other challenges, we made the strategic decision to transition to Together AI. While this transition was beneficial in many ways, it also presented its own set of challenges, such as adapting our existing infrastructure and workflows to a new platform.

### Accomplishments we're proud of

Achieving a seamless blend of generated text and images that genuinely aid users in understanding complex tasks is our hallmark. The intuitive interface and the reliability of our guides have garnered positive early feedback, reinforcing our belief in the platform's potential.

### What we learned

The project was a deep dive into the intricacies of text and image generation technologies. We learned about different LLMs and Stable Diffusion models. We learned how to deploy a backend and a frontend. We learned about prompt engineering, tokenization and token count, and much more!

### What's next

Our vision is to continually refine and expand the range of topics available. We're exploring the integration of videos and interactive elements, and we're also looking into fostering a community where users can contribute, modify, and rate guides, making "How To ANYTHING" a collaborative knowledge hub powered by Generative AI.

## README (from the GitHub repository)

This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app).

## Getting Started

First, run the development server:

```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```

Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.

You can start editing the page by modifying `app/page.js`. The page auto-updates as you edit the file.

This project uses [`next/font`](https://nextjs.org/docs/basic-features/font-optimization) to automatically optimize and load Inter, a custom Google Font.

## Learn More

To learn more about Next.js, take a look at the following resources:

- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.

You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js/) - your feedback and contributions are welcome!

## Deploy on Vercel

The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.

Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.


## Detected evidence (automated analysis)

Indexed codebase: 21 recognized source files, 28 KB.
- CSS (language) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code
- FastAPI (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (30 of 30)

```
.gitignore
backend/.eslintrc.json
backend/.gitignore
backend/api/index.py
backend/app/globals.css
backend/app/layout.tsx
backend/app/page.tsx
backend/next.config.js
backend/package.json
backend/pipeline.ipynb
backend/postcss.config.js
backend/README.md
backend/requirements.txt
backend/static/styles_search.css
backend/static/styles.css
backend/tailwind.config.js
backend/templates/index.html
backend/templates/search.html
backend/tsconfig.json
backend/utils/__init__.py
backend/utils/description_to_image.py
backend/utils/get_gpt_response.py
backend/zillisDB.py
jsconfig.json
next.config.js
package.json
pages/_app.js
pages/index.js
README.md
styles/main.css
```

### Dependencies

- backend/package.json: @types/node@20.2.4, @types/react@18.2.7, @types/react-dom@18.2.4, autoprefixer@10.4.14, concurrently@^8.0.1, eslint@8.41.0, eslint-config-next@13.4.4, next@13.4.4, postcss@8.4.23, react@18.2.0, react-dom@18.2.0, tailwindcss@3.3.2, typescript@5.0.4
- backend/requirements.txt: fastapi[all]@==0.100.1, ipython@==8.14.0, Jinja2@==3.1.2, openai@==0.27.6, Pillow@==9.5.0, python-dotenv@==1.0.0, starlette@==0.27.0, together@==0.2.6, uvicorn[standard]@==0.23.2
- package.json: @types/node@20.2.4, @types/react@18.2.7, @types/react-dom@18.2.4, autoprefixer@10.4.14, concurrently@^8.0.1, eslint@8.41.0, eslint-config-next@13.4.4, next@14.0.1, postcss@8.4.23, react@^18, react-dom@^18, react-router-dom@^6.18.0, tailwindcss@3.3.2, typescript@5.0.4

### Recent commits (newest first)

- revamped everything
- fixed npm build
- saving merge
- added deployment
- added search.html
- second commit
- Nathan's first commit
- added together
- updated requirements.txt
- overriding
- got proof of concept to work
- rip my keys
- update readme
- first commit
- deleted license
- updated requirements again
- updated requirements
- added database
- changed response
- changed README

## Key source files (fetched from GitHub, selected and truncated for size)

### package.json

```
{
    "name": "my-project",
    "version": "0.1.0",
    "private": true,
    "scripts": {
      "dev": "next dev",
      "build": "next build",
      "start": "next start",
      "lint": "next lint",
      "fastapi-dev": "pip3 install -r backend/requirements.txt && python3 -m uvicorn backend.api.index:app --reload"
    },
    "dependencies": {
      "@types/node": "20.2.4",
      "@types/react": "18.2.7",
      "@types/react-dom": "18.2.4", 
      "autoprefixer": "10.4.14",
      "concurrently": "^8.0.1",
      "eslint": "8.41.0",
      "eslint-config-next": "13.4.4",
      "next": "14.0.1",
      "postcss": "8.4.23",
      "react": "^18",
      "react-dom": "^18",
      "react-router-dom": "^6.18.0",
      "tailwindcss": "3.3.2",
      "typescript": "5.0.4"
    }
  }
  
```

### backend/requirements.txt

```
fastapi[all]==0.100.1
uvicorn[standard]==0.23.2
python-dotenv==1.0.0

starlette==0.27.0
Jinja2==3.1.2
Pillow==9.5.0
openai==0.27.6
ipython==8.14.0
together==0.2.6
```

### backend/package.json

```
{
  "name": "nextjs-fastapi",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "fastapi-dev": "pip3 install -r requirements.txt && python3 -m uvicorn api.index:app --reload",
    "next-dev": "next dev",
    "dev": "concurrently \"npm run next-dev\" \"npm run fastapi-dev\"",
    "build": "npm run dev",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@types/node": "20.2.4",
    "@types/react": "18.2.7",
    "@types/react-dom": "18.2.4", 
    "autoprefixer": "10.4.14",
    "concurrently": "^8.0.1",
    "eslint": "8.41.0",
    "eslint-config-next": "13.4.4",
    "next": "13.4.4",
    "postcss": "8.4.23",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "tailwindcss": "3.3.2",
    "typescript": "5.0.4"
  }
}
 
```

### pages/index.js

```javascript
import { useState } from "react";

export default function Home() {
    const [query, setQuery] = useState("");
    const [results, setResults] = useState({ title: "", steps: [] });
    const [loading, setLoading] = useState(false);
    const [showResults, setShowResults] = useState(false);

    const fetchResults = async () => {
        setLoading(true);
        setShowResults(false);
        try {
            const response = await fetch(
                `http://localhost:8000/query?query=${query}`
            );
            const data = await response.json();
            setResults(data);
            setLoading(false);
            setShowResults(true);
        } catch (error) {
            console.error("Error fetching data:", error);
            setLoading(false);
        }
    };

    const handleQueryChange = (e) => {
        setQuery(e.target.value);
    };

    const handleSubmit = (e) => {
        e.preventDefault();
        fetchResults();
    };

    return (
        <div className="container">
            <Navbar />
            {loading ? (
                <p>Loading...</p>
            ) : showResults ? (
                <ResultsList title={results.title} steps={results.steps} />
            ) : (
                <MainContent
                    query={query}
                    onQueryChange={handleQueryChange}
                    onSubmit={handleSubmit}
                />
            )}
            <Footer />
        </div>
    );
}

const Navbar = () => (
    <div id="navbar">
        <a href="/">Home</a>
        <a href="/about">About</a>
        <a href="/development">Development</a>
    </div>
);

const MainContent = ({ query, onQueryChange, onSubmit, results }) => (
    <div id="main-content">
        <SearchForm
            query={query}
            onChange={onQueryChange}
            onSubmit={onSubmit}
        />
        {results && results.title && (
            <ResultsList title={results.title} steps={results.steps} />
        )}
    </div>
);

const SearchForm = ({ query, onChange, onSubmit }) => (
    <form onSubmit={onSubmit}>
        <input
            type="text"
            id="query"
            name="query"
            placeholder="How to:"
            value={query}
            onChange={onChange}
        />
        <input type="submit" value="Submit" />
    </form>
);

const ResultsList = ({ title, steps }) => (
    <div>
        <h1>{title}</h1>
        <ul>
            {steps.map((item, index) => (
                <li key={index}>
                    {item.instruction}
                    <img
                        src={`data:image/png;base64,${item.image_path}`}
                        alt={`Image for ${item.instruction}`}
                    />
                </li>
            ))}
        </ul>
    </div>
);

const Footer = () => (
    <footer>
        <div className="container">
            <p>How To Dummies</p>
            <p>&copy; 2023 Your Company Name. All rights reserved.</p>
        </div>
    </footer>
);

```

### backend/app/layout.tsx

```typescript
import './globals.css'
import { Inter } from 'next/font/google'

const inter = Inter({ subsets: ['latin'] })

export const metadata = {
  title: 'Create Next App',
  description: 'Generated by create next app',
}

export default function RootLayout({
  children,
}: {
  children: React.ReactNode
}) {
  return (
    <html lang="en">
      <body className={inter.className}>{children}</body>
    </html>
  )
}

```

### backend/api/index.py

```python
from fastapi import FastAPI, Request, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import HTMLResponse
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from backend.utils.get_gpt_response import get_gpt_response
from backend.utils.description_to_image import description_to_image

app = FastAPI()
app.mount("/static", StaticFiles(directory="backend/static"), name="static")

ALLOWED_ORIGINS = ["http://localhost:3000"]
TEMPLATES = Jinja2Templates(directory="templates")

app.add_middleware(
    CORSMiddleware,
    allow_origins=ALLOWED_ORIGINS,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

@app.get("/", response_class=HTMLResponse)
async def root(request: Request): 
    return TEMPLATES.TemplateResponse("search.html", {"request": request})

@app.get("/query")
async def query_instructions(query: str):
    print(query)
    try:
        text_response = get_gpt_response(query)
        title, *steps = text_response.split(':')
        title = title.split('\n')[0]

        parsed_data = parse_steps(steps)
        return {
            "title": title,
            "steps": parsed_data
        }
    except Exception as e:
        raise HTTPException(status_code=400, detail="Error processing query")

def parse_steps(steps: list) -> list:
    parsed_data = []
    for idx, step in enumerate(steps, start=1):
        segments = step.split('<image>', 1)  
        
        if len(segments) != 2:
            continue

        instruction, description = segments
        if not instruction.startswith("Step"):
            instruction = f'Step {idx}:{instruction}'
        else:
            instruction = f'Step {idx}: {instruction.split(":", 1)[1]}'
        
        print(f'instructions {instruction}')
        print(f'description {description}')
        parsed_data.append({
            "instruction": instruction.strip(),
            "image_path": description_to_image(description.split("<image>")[0].strip())
        })
    return parsed_data


```

### backend/app/page.tsx

```typescript
import Image from "next/image";
import Link from "next/link";

export default function Home() {
  return (
    <main className="flex min-h-screen flex-col items-center justify-between p-24">
      <div className="z-10 w-full max-w-5xl items-center justify-between font-mono text-sm lg:flex">
        <p className="fixed left-0 top-0 flex w-full justify-center border-b border-gray-300 bg-gradient-to-b from-zinc-200 pb-6 pt-8 backdrop-blur-2xl dark:border-neutral-800 dark:bg-zinc-800/30 dark:from-inherit lg:static lg:w-auto  lg:rounded-xl lg:border lg:bg-gray-200 lg:p-4 lg:dark:bg-zinc-800/30">
          Get started by editing&nbsp;
          <Link href="/api/python">
            <code className="font-mono font-bold">api/index.py</code>
          </Link>
        </p>
        <div className="fixed bottom-0 left-0 flex h-48 w-full items-end justify-center bg-gradient-to-t from-white via-white dark:from-black dark:via-black lg:static lg:h-auto lg:w-auto lg:bg-none">
          <a
            className="pointer-events-none flex place-items-center gap-2 p-8 lg:pointer-events-auto lg:p-0"
            href="https://vercel.com?utm_source=create-next-app&utm_medium=appdir-template&utm_campaign=create-next-app"
            target="_blank"
            rel="noopener noreferrer"
          >
            By{" "}
            <Image
              src="/vercel.svg"
              alt="Vercel Logo"
              className="dark:invert"
              width={100}
              height={24}
              priority
            />
          </a>
        </div>
      </div>

      <div className="relative flex place-items-center before:absolute before:h-[300px] before:w-[480px] before:-translate-x-1/2 before:rounded-full before:bg-gradient-radial before:from-white before:to-transparent before:blur-2xl before:content-[''] after:absolute after:-z-20 after:h-[180px] after:w-[240px] after:translate-x-1/3 after:bg-gradient-conic after:from-sky-200 after:via-blue-200 after:blur-2xl after:content-[''] before:dark:bg-gradient-to-br before:dark:from-transparent before:dark:to-blue-700 before:dark:opacity-10 after:dark:from-sky-900 after:dark:via-[#0141ff] after:dark:opacity-40 before:lg:h-[360px]">
        <Image
          className="relative dark:drop-shadow-[0_0_0.3rem_#ffffff70] dark:invert"
          src="/next.svg"
          alt="Next.js Logo"
          width={180}
          height={37}
          priority
        />
      </div>

      <div className="mb-32 grid text-center lg:mb-0 lg:grid-cols-4 lg:text-left">
        <a
          href="https://nextjs.org/docs?utm_source=create-next-app&utm_medium=appdir-template&utm_campaign=create-next-app"
          className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"
          target="_blank"
          rel="noopener noreferrer"
        >
          <h2 className={`mb-3 text-2xl font-semibold`}>
            Docs{" "}
            <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">
              -&gt;
            </span>
          </h2>
          <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>
            Find in-depth information about Next.js features and API.
          </p>
        </a>

        <a
          href="https://nextjs.org/learn?utm_source=create-next-app&utm_medium=appdir-template-tw&utm_campaign=create-next-app"
          className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800 hover:dark:bg-opacity-30"
          target="_blank"
          rel="noopener noreferrer"
        >
          <h2 className={`mb-3 text-2xl font-semibold`}>
            Learn{" "}
            <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">
              -&gt;
            </span>
          </h2>
          <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>
            Learn about Next.js in an interactive course with&nbsp;quizzes!
          </p>
        </a>

        <a
          href="https://vercel.com/templates?framework=next.js&utm_source=create-next-app&utm_medium=appdir-template&utm_campaign=create-next-app"
          className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"
          target="_blank"
          rel="noopener noreferrer"
        >
          <h2 className={`mb-3 text-2xl font-semibold`}>
            Templates{" "}
            <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">
              -&gt;
            </span>
          </h2>
          <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>
            Explore the Next.js 13 playground.
          </p>
        </a>

        <a
          href="https://vercel.com/new?utm_source=create-next-app&utm_medium=appdir-template&utm_campaign=create-next-app"
          className="group rounded-lg border border-transparent px-5 py-4 transition-colors hover:border-gray-300 hover:bg-gray-100 hover:dark:border-neutral-700 hover:dark:bg-neutral-800/30"
          target="_blank"
          rel="noopener noreferrer"
        >
          <h2 className={`mb-3 text-2xl font-semibold`}>
            Deploy{" "}
            <span className="inline-block transition-transform group-hover:translate-x-1 motion-reduce:transform-none">
              -&gt;
            </span>
          </h2>
          <p className={`m-0 max-w-[30ch] text-sm opacity-50`}>
            Instantly deploy your Next.js site to a shareable URL with Vercel.
          </p>
        </a>
      </div>
    </main>
  );
}

```

### next.config.js

```javascript
/** @type {import('next').NextConfig} */
const nextConfig = {}

module.exports = nextConfig

```

### backend/postcss.config.js

```javascript
module.exports = {
    plugins: {
        tailwindcss: {},
        autoprefixer: {},
    },
};

```

### pages/_app.js

```javascript
import '../styles/main.css';

function MyApp({ Component, pageProps }) {
  return <Component {...pageProps} />;
}

export default MyApp;

```

[11 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]