# Project export: InsightBot

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: A personalized study tool powered by Retrieval Augmented Generation
- Devpost: https://devpost.com/software/insightbot
- GitHub: https://github.com/kyileiaye2021/insightbot
- Team: 4 GitHub contributor(s) — Kyi Lei Aye (She/ Her) (20 commits), Thomas Rodriguez (13 commits), Sam (8 commits), alexmenchtchikov (2 commits)

## Devpost submission (written by the team)

### Inspiration

The inspiration for Insightbot came from the growing demand for personalized learning and productivity tools that can support users in managing their tasks and goals more effectively. As students, we often found ourselves struggling to balance multiple responsibilities, while searching for tailored information or assistance at critical moments. We realized that there was a need for a tool that not only helps users stay organized with to-do lists and goals tracking but also provides on-demand learning support based on their personal data, like notes or resources. This inspired us to combine the power of Retrieval-Augmented Generation (RAG) with a productivity suite to create a seamless and personalized learning experience, accessible anytime.

### What it does

InsightBot is an intelligent study tool powered by RAG that delivers a personalized learning experience, helping users achieve their goals. Our platform offers multiple tools that helps users to stay organized, achieve their goals, and receive tailored support to overcome learning challenges, ensuring continuous progress at any time.

### How we built it

Using NextJS, we started our project with UI/UX development. Following, we integrated an AI chatbot with Open AI API key and downstreamed the chatbot with RAG by utilizing Open AI embeddings, Pinecone database, and Pinecone API. Then, with Python, we developed backend and handled file upload, to do list, and goal tracker features.

### Challenges we ran into

We encountered an issue integrating RAG into the app due to recent updates in OpenAI's embedding documentation. To resolve this, we needed to update the code to align with the latest OpenAI API for vector embeddings.

### Accomplishments we're proud of

We're proud of successfully integrating RAG within the chatbot and the goal tracker. The chatbot provides personalized learning assistance based on user-uploaded documents, while the goal tracker provides information on whether the added objectives are SMART (Specific, Measurable, Achievable, Relevant, and Time-bound). Overcoming the challenge of adapting to the updated OpenAI API for embeddings was a significant accomplishment. Additionally, we developed a seamless UI/UX offering users an intuitive and productive experience.

### What we learned

Throughout this project, we deepened our understanding of RAG and how to effectively integrate it with APIs like OpenAI and Pinecone. We also learned to adapt quickly to changes in documentation and APIs, improving our problem-solving abilities. On the front-end, we gained valuable experience in creating user-friendly interfaces with NextJS, while on the back-end, we sharpened our skills in database management and API integration. Additionally, collaboration taught us how to manage our time effectively and prioritize tasks under tight deadlines.

### What's next

for Insightbot We plan to enhance Insightbot by allowing user to upload more than one file, and supporting more data types like videos and links. We’re also exploring how to create tasks for the to-do list automatically based on a selected SMART goal using Open AI's API.

## README (from the GitHub repository)

# Insightbot

## Project Description
InsightBot is an intelligent study tool powered by **RAG** that delivers a personalized learning experience, helping users achieve their goals. Our platform offers multiple tools that helps users to stay organized, achieve their goals, and receive tailored support to overcome learning challenges, ensuring continuous progress at any time.

![Home Page](./readme_img/home.png) 
![Main Page](./readme_img/main.png) 

## Key Features
This AI personal web app combines:
- To-do list
- Goal Tracker
- AI-powered RAG chatbot

## Technologies Used
- Python
- NextJS
- PINECONE
- TYPESCRIPT
- OPEN AI API

## Usage
After navigating to the main page, users can create To-Do List. Click on the ‘+’ button to create a new task. They can set the task’s name. On the main page, users can create their personal or academic goals in the Goal List section. 

Moreover, they can go to the AI Chatbot section to ask questions and get personalized assistance. They can upload files (PDFs, notes, etc.) by clicking the upload button. These files will be analyzed to help the chatbot provide relevant answers. After uploading the files, users can ask the chatbot any question related to your study materials or tasks. It will retrieve information from the files and provide tailored responses. They can also use the chatbot for on-demand assistance, whether they are stuck on a concept or need help in the materials outside of the database.

## Future Directions
We plan to enhance Insightbot by supporting more diverse types of data uploads, including videos and links, allowing users to interact with a broader range of content. We’re also exploring how to refine the chatbot’s natural language processing capabilities to handle more complex queries and improve user experience.


## Detected evidence (automated analysis)

Indexed codebase: 7 recognized source files, 25 KB.
- CSS (language) — detected in the code
- Flask (technology) — detected in the code
- LangChain (technology) — detected in the code
- Next.js (technology) — detected in the code
- OpenAI (technology) — detected in the code
- Python (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- TypeScript (language) — detected in the code

## Codebase structure (from repository index)

### Files (14 of 14)

```
.eslintrc.json
.gitignore
next.config.mjs
package.json
postcss.config.mjs
README.md
requirements.txt
setup_rag.py
src/app/dashboard/page.tsx
src/app/globals.css
src/app/layout.tsx
src/app/page.tsx
tailwind.config.ts
tsconfig.json
```

### Dependencies

- package.json: @emotion/react@^11.13.3, @emotion/styled@^11.13.0, @mui/icons-material@^6.1.4, @mui/material@^6.1.4, @pinecone-database/pinecone@^3.0.3, @types/node@^20, @types/react@^18, @types/react-dom@^18, @vercel/analytics@^1.3.1, axios@^1.7.7, eslint@^8, eslint-config-next@14.2.15, langchain@^0.3.2, multer@^1.4.5-lts.1, next@14.2.15, openai@^4.68.1, postcss@^8, react@^18, react-dom@^18, tailwindcss@^3.4.1, tiktoken@^1.0.17, typescript@^5
- requirements.txt: annotated-types@==0.7.0, anyio@==4.6.2.post1, blinker@==1.8.2, certifi@==2024.8.30, click@==8.1.7, distro@==1.9.0, Flask@==3.0.3, Flask-Cors@==5.0.0, h11@==0.14.0, httpcore@==1.0.6, httpx@==0.27.2, idna@==3.10, itsdangerous@==2.2.0, Jinja2@==3.1.4, jiter@==0.6.1, MarkupSafe@==3.0.2, numpy@==1.26.4, openai@==1.52.0, pinecone-client@==5.0.1, pinecone-plugin-inference@==1.1.0, pinecone-plugin-interface@==0.0.7, pydantic@==2.9.2, pydantic_core@==2.23.4, PyPDF2@==3.0.1, sniffio@==1.3.1, tiktoken@==0.5.0, tqdm@==4.66.5, typing_extensions@==4.12.2, urllib3@==2.2.3, Werkzeug@==3.0.4

### Recent commits (newest first)

- Merge pull request #10 from kyileiaye2021/thomas
- Added AI generated todo tasks
- Update README.md
- Merge branch 'main' of https://github.com/kyileiaye2021/insightbot
- updated setup_rag.py
- Merge pull request #9 from kyileiaye2021/thomas
- UI fixes
- Added photos in readme
- Updated Readme file
- Updated Readme
- Merge branch 'main' of https://github.com/kyileiaye2021/insightbot
- last update
- Merge pull request #8 from kyileiaye2021/sam-dev
- Todo Function + enhancements
- Merge pull request #7 from kyileiaye2021/sam-dev
- Goal Advise functionality
- Merge pull request #6 from kyileiaye2021/sam-dev
- Adding goal advise
- Merge pull request #5 from kyileiaye2021/sam-dev
- sam test

## Key source files (fetched from GitHub, selected and truncated for size)

### requirements.txt

```
annotated-types==0.7.0
anyio==4.6.2.post1
blinker==1.8.2
certifi==2024.8.30
click==8.1.7
distro==1.9.0
Flask==3.0.3
Flask-Cors==5.0.0
h11==0.14.0
httpcore==1.0.6
httpx==0.27.2
idna==3.10
itsdangerous==2.2.0
Jinja2==3.1.4
jiter==0.6.1
MarkupSafe==3.0.2
openai==1.52.0
numpy==1.26.4
pinecone-client==5.0.1
pinecone-plugin-inference==1.1.0
pinecone-plugin-interface==0.0.7
pydantic==2.9.2
pydantic_core==2.23.4
PyPDF2==3.0.1
sniffio==1.3.1
tqdm==4.66.5
typing_extensions==4.12.2
urllib3==2.2.3
Werkzeug==3.0.4
tiktoken==0.5.0

```

### package.json

```
{
  "name": "insightbot",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint"
  },
  "dependencies": {
    "@emotion/react": "^11.13.3",
    "@emotion/styled": "^11.13.0",
    "@mui/icons-material": "^6.1.4",
    "@mui/material": "^6.1.4",
    "@pinecone-database/pinecone": "^3.0.3",
    "@vercel/analytics": "^1.3.1",
    "axios": "^1.7.7",
    "langchain": "^0.3.2",
    "multer": "^1.4.5-lts.1",
    "next": "14.2.15",
    "openai": "^4.68.1",
    "react": "^18",
    "react-dom": "^18",
    "tiktoken": "^1.0.17"
  },
  "devDependencies": {
    "@types/node": "^20",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "eslint": "^8",
    "eslint-config-next": "14.2.15",
    "postcss": "^8",
    "tailwindcss": "^3.4.1",
    "typescript": "^5"
  }
}

```

### src/app/layout.tsx

```typescript
import type { Metadata } from "next";
import localFont from "next/font/local";
import "./globals.css";

const geistSans = localFont({
  src: "./fonts/GeistVF.woff",
  variable: "--font-geist-sans",
  weight: "100 900",
});
const geistMono = localFont({
  src: "./fonts/GeistMonoVF.woff",
  variable: "--font-geist-mono",
  weight: "100 900",
});

export const metadata: Metadata = {
  title: "Create Next App",
  description: "Generated by create next app",
};

export default function RootLayout({
  children,
}: Readonly<{
  children: React.ReactNode;
}>) {
  return (
    <html lang="en">
      <body
        className={`${geistSans.variable} ${geistMono.variable} antialiased`}
      >
        {children}
      </body>
    </html>
  );
}

```

### src/app/page.tsx

```typescript
'use client'

import { Box, Button, Stack, Typography } from '@mui/material'
import { useState } from 'react'

export default function Home() {
  return (
    <Box
      width="100vw"
      height="100vh"
      display="flex"
      flexDirection="column"
      justifyContent="flex-start"
      alignItems="center"
      p={2}
    >
      <Stack
        width="50vw"
        spacing={0}
        p={4}
        style={{ border: '4px solid #004d80', borderRadius: '5px', backgroundColor: 'rgba(0, 77, 128, 0.3)' }}
        alignItems="center"
      >
        <Typography style={{ color: '#bfbfbf', fontSize: '2rem', textAlign: 'center' }}><b>Welcome to</b></Typography>
        <Typography style={{ color: '#0099ff', fontSize: '4rem', textAlign: 'center' }}><b>InsightBot</b></Typography>

        <Button
          type="submit"
          variant="contained"
          fullWidth
          onClick={() => window.location.href = '/dashboard'}
        >Go to dashboard</Button>
      </Stack>
    </Box>
  )
}

```

### src/app/dashboard/page.tsx

```typescript
'use client'
import { Box, Button, Stack, TextField, Typography, Paper, IconButton, Dialog, DialogTitle, DialogContent, DialogActions, Checkbox, AppBar, Toolbar } from '@mui/material'
import { useState } from 'react'
import axios from 'axios'
import SendIcon from '@mui/icons-material/Send'
import AddIcon from '@mui/icons-material/Add'

export default function Home() {
  const [file, setFile] = useState(null) // State to store selected file
  const [messages, setMessages] = useState([
    {
      role: 'assistant',
      content: ' Hi! How can I help you today?',
    },
  ])
  const [message, setMessage] = useState('')
  const [uploadMessage, setUploadMessage] = useState('') // State for showing upload status
  const [goals, setGoals] = useState([])
  const [newGoal, setNewGoal] = useState('')
  const [openDialog, setOpenDialog] = useState(false)
  const [selectedGoal, setSelectedGoal] = useState('')
  const [goalAdvice, setGoalAdvice] = useState('')
  const [generatedTasks, setGeneratedTasks] = useState('')
  const [todos, setTodos] = useState([])
  const [newTodo, setNewTodo] = useState('')

  // Functioin to handle file selection
  //This function is triggered when a user selects a file
  const handleFileChange = (e) => {
    setFile(e.target.files[0])
  }

  //function to handle file upload
  //This function is triggered when the user clicks the Upload Files button. 
  const handleUpload = async () => {
    if (!file) {
      setUploadMessage('Please select a file to upload')
      return
    }
    const formData = new FormData()
    formData.append('file', file)

    try {
      const response = await axios.post('http://127.0.0.1:5000/upload', formData)  // URL to your Flask backend
      setUploadMessage(response.data.message)  // Show success message
    } catch (error) {
      setUploadMessage('Error uploading file, try again')  // Show error message
    }
  }

  const sendMessage = async () => {
    setMessage('')
    setMessages((messages) => [
      ...messages,
      { role: 'user', content: message },
      { role: 'assistant', content: '' },
    ])

    const response = fetch('http://127.0.0.1:5000/chat', { //// Point to Flask backend
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({ question: message }), // send only the user message
    }).then(async (res) => {
      const reader = res.body.getReader()
      const decoder = new TextDecoder()
      let result = ''

      return reader.read().then(function processText({ done, value }) {
        if (done) {
          return result
        }

        const text = decoder.decode(value || new Uint8Array(), { stream: true })
        setMessages((messages) => {
          let lastMessage = messages[messages.length - 1]
          let otherMessages = messages.slice(0, messages.length - 1)
          return [
            ...otherMessages,
            { ...lastMessage, content: lastMessage.content + text },
          ]
        })
        return reader.read().then(processText)
      })
    })
  }

  const addGoal = () => {
    if (newGoal.trim()) {
      setGoals([...goals, newGoal.trim()])
      setNewGoal('')
    }
  }

  const getGoalAdvice = async (goal) => {
    try {
      const response = await fetch('http://127.0.0.1:5000/goaladvise', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ goal: goal }),
      })

      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`)
      }

      const data = await response.json()
      setGoalAdvice(data.advice)
      setOpenDialog(true)
    } catch (error) {
      console.error('Error:', error)
      setGoalAdvice('An error occurred while fetching goal advice.')
      setOpenDialog(true)
    }
  }

  const addTodo = () => {
    if (newTodo.trim()) {
      setTodos([...todos, { text: newTodo.trim(), done: false }])
      setNewTodo('')
    }
  }

  const toggleTodo = (index) => {
    const newTodos = [...todos]
    newTodos[index].done = !newTodos[index].done
    setTodos(newTodos)
  }

  const createTasksGoal = async () => {
    try {
      const response = await fetch('http://127.0.0.1:5000/generatetodotask', {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ goal: selectedGoal }),
      })

      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`)
      }

      const data = await response.json()

      if (data.generated.tasks && Array.isArray(data.generated.tasks)) {
        for (let i = 0; i < data.generated.tasks.length; i++) {
            setTodos((prevTodos) => [...prevTodos, { text: data.generated.tasks[i], done: false }])
        }
        setOpenDialog(false)
      } else {
        console.log(data.generated);
      }
    } catch (error) {
      console.error('Error:', error)
      setGeneratedTasks('An error occurred while creating tasks.')
      setOpenDialog(false)
    }
  }

  return (
    <Box
      width="100vw"
      height="100vh"
      display="flex"
      flexDirection="column"
      justifyContent="flex-start"
      alignItems="center"
    >
      <AppBar position="static" sx={{ bgcolor: '#004d80' }}>
        <Toolbar>
          <Typography variant="h4" component="div" sx={{ flexGrow: 1, textAlign: 'center' }}>
            INSIGHTBOT - AI Student Assistant
          </Typography>
        </Toolbar>
      </AppBar>

      {/* Upload Button */}
      <Stack width="100vw" p={2} alignItems="center" style={{
        backgroundColor: '#004d80',
      }}>
        <Typography><b>Select a PDF file to add to the database</b></Typography>
        <Stack direction="row" spacing={1}>
          <input type="file" accept=".pdf" onChange={handleFileChange} name='file' style={{ border: '2px dashed #0099ff', borderRadius: '5px', padding: '5px' }} />
          <Button variant="contained" onCli
[truncated — 7531 more characters]
```

### tailwind.config.ts

```typescript
import type { Config } from "tailwindcss";

const config: Config = {
  content: [
    "./src/pages/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/components/**/*.{js,ts,jsx,tsx,mdx}",
    "./src/app/**/*.{js,ts,jsx,tsx,mdx}",
  ],
  theme: {
    extend: {
      colors: {
        background: "var(--background)",
        foreground: "var(--foreground)",
      },
    },
  },
  plugins: [],
};
export default config;

```

### setup_rag.py

```python
from flask import Flask, request, jsonify 
from flask_cors import CORS 
from werkzeug.utils import secure_filename
from dotenv import load_dotenv
import os
from pinecone import Pinecone, ServerlessSpec
import openai
import PyPDF2
import tiktoken
import json

app = Flask(__name__)
CORS(app, resources={r"/*": {"origins": "http://localhost:3000"}})

# load environment variables
load_dotenv(dotenv_path='.env.local')

# initialize pinecone
pc = Pinecone(api_key=os.getenv("PINECONE_API_KEY"))

# Create Pinecone index if not exists
if "rag" not in pc.list_indexes().names():
    pc.create_index(
        name="rag",
        dimension=1536,
        metric="cosine",
        spec=ServerlessSpec(cloud='aws', region="us-east-1")
    )

index = pc.Index("rag")

# Set openAI API key
openai.api_key = os.getenv("OPENAI_API_KEY")

# Func to create embeddings
def create_embedding(text):
    client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    response = client.embeddings.create(input=text, model='text-embedding-3-small')
    return response.data[0].embedding

# def create_embedding(text):
#     response = openai.Embedding.create(input=text, model='text-embedding-ada-002')
#     return response['data'][0]['embedding']

# func to split text into chunks
def split_text(text, max_tokens=800):
    tokenizer = tiktoken.get_encoding('p50k_base') # This encoding is suitable for GPT-3.5 models
    tokens = tokenizer.encode(text)
    
    chunks = []
    for i in range(0, len(tokens), max_tokens):
        chunk = tokens[i:i+max_tokens]
        chunks.append(tokenizer.decode(chunk))
    return chunks

# func to extract text from a PDF
def extract_text_from_pdf(pdf_file):
    reader = PyPDF2.PdfReader(pdf_file)
    text = ""
    # extract text from each page
    for page_num in range(len(reader.pages)):
        page = reader.pages[page_num]
        text += page.extract_text()
        
    return text

# Route to handle file uploaad and embedding creation
@app.route('/upload', methods=['POST'])
def upload_file():
    try:
        if 'file' not in request.files:
            return jsonify({"error": "No file part"}), 400
        
        file = request.files['file']
        filename = secure_filename(file.filename)
        
        # check if the uploaded file is a PDF
        if not filename.endswith('.pdf'):
            return jsonify({"error": "Only PDF files are allowed"}), 400
        
        # Extract text from the PDF file
        file_text = extract_text_from_pdf(file)
    
        # Split text into smaller chunks
        chunks = split_text(file_text)
        
        # Generate embeddin for the extracted text
        #embedding = create_embedding(chunks)
        
        # generate embeddings for each chunk
        embeddings = []
        for chunk in chunks:
            embedding = create_embedding(chunk)
            embeddings.append({
                "id": f"{filename}_chunk_{chunks.index(chunk)}",
                "values": embedding,
                "metadata": {"filename": filename, 'text':chunk}
            })
        
        # Store the embedding in Pinecone
        index.upsert(vectors=embeddings)
        
        return jsonify({'message': f'File {filename} uploaded and embedding stored in Pinecone'}), 200
    except Exception as e:
        print(f"Error: {str(e)}")
        return jsonify({"error": str(e)}), 500
    
# for handling user queries
@app.route('/chat', methods=['POST'])
def chat():
    user_question = request.json.get('question')
    if not user_question:
        return jsonify({"error": "No question provided"}), 400
    
    # Generate an embedding for the user's question
    question_embedding = create_embedding(user_question)
    
    # Query pinecone for the most relevant data
    query_response = index.query(
        vector=question_embedding,
        top_k=5,  # Retrieve top 5 relevant matches
        include_metadata=True
    )
    
    # Construct a context from the most relevant matches
    context = "\n".join([match['metadata']['text'] for match in query_response['matches']])
    
    # generate a response using OpenAI
    client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": f"Context: {context}\nQuestion: {user_question}"}
        ],
        max_tokens=150
    )
    
    return jsonify({"answer": response.choices[0].message.content}), 200

# new route
@app.route('/goaladvise', methods=['POST'])
def goal_advise():
    goal_statement = request.json.get('goal')
    if not goal_statement:
        return jsonify({"error": "No goal statement provided"}), 400
    
    # Sample context (you can edit this later)
    context = """
    Effective goal-setting involves creating SMART goals: Specific, Measurable, Achievable, Relevant, and Time-bound.
    Consider breaking larger goals into smaller, manageable tasks.
    Regular review and adjustment of goals is important for long-term success.
    ONLY provide advice on the goal if it is an ACADEMIC goal, related to Education, Career, or Personal Development.
    If it is not, please respond with "I am sorry, I can only provide advice on academic goals."
    """
    
    # Generate a response using OpenAI
    client = openai.OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
    response = client.chat.completions.create(
        model='gpt-4o-mini',
        messages=[
            {"role": "system", "content": "You are a helpful assistant specializing in goal-setting and personal development."},
            {"role": "user", "content": f"Context: {context}\nGoal Statement: {goal_statement}\n Check and confirm whether the goal is SMART. Please provide advice on this goal. Be short and concise."}
        ],
        max_tokens=200
    )
    
    return jsonify({"advice": response.choices[0].message.content}), 200

# generate todo t
[truncated — 1287 more characters]
```

### src/app/globals.css

```css
@tailwind base;
@tailwind components;
@tailwind utilities;

:root {
  --background: #ffffff;
  --foreground: #171717;
}

@media (prefers-color-scheme: dark) {
  :root {
    --background: #0a0a0a;
    --foreground: #ededed;
  }
}

body {
  color: var(--foreground);
  background: var(--background);
  font-family: Arial, Helvetica, sans-serif;
}

@layer utilities {
  .text-balance {
    text-wrap: balance;
  }
}

```