# Project export: Study Vault

This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small.

## Project metadata

- Hackathon: Cal Hacks 11.0
- Tagline: Your All-in-One Study Hub: Organize, Review, Succeed!
- Devpost: https://devpost.com/software/study-vault
- GitHub: https://github.com/devanshalok/StudyVault
- Team: 1 GitHub contributor(s) — Devansh Alok (6 commits)

## Devpost submission (written by the team)

### Inspiration

canvas chrome extension

### What it does

summarization assistant using AI

### How we built it

next js for backend and react js for frontend

### Challenges we ran into

integration of AI and storing files in the database

### Accomplishments we're proud of

searching function

### What we learned

Learned how to integrate hugging face speech-to-text API and how to build connection between the frontend component to the backend api calling.

### What's next

video incorporation and getting information through the video transcript

## README (from the GitHub repository)

# Study Vault

**Study Vault** is a web application designed to help students organize and access their lecture materials more efficiently. With Study Vault, students can upload various types of study resources like PDFs, videos, and photos, and easily search for specific terms or concepts within these materials. The app also provides summarization features, making it easier for students to review and prepare for exams.

## Project Overview

Study Vault aims to be a centralized resource hub where students can upload, search, and review academic materials quickly and effectively. It combines document management with search and summarization capabilities, making studying a more streamlined experience.

## Getting Started

Follow these steps to set up a local copy of the project for development, testing, and experimentation.

### Prerequisites

To run Study Vault locally, ensure you have:

- **Node.js** (v12 or higher)
- **npm** or **yarn** for managing dependencies
- **Python** for backend processing (optional, if used)
- Basic knowledge of **HTML**, **CSS**, and **JavaScript**

#### Installation Example

1. **Clone the Repository**

   ```bash
   git clone https://github.com/devanshalok/StudyVault.git
   ```

2. **Navigate to the Directory**

   ```bash
   cd study-vault
   ```

3. **Install Dependencies**

   ```bash
   npm install
   ```

### Installing

To set up Study Vault on your local machine:

1. **Backend Setup** (if applicable): Configure any backend services, such as document parsing or summarization APIs.
2. **Frontend Setup**: Run the development server to access the app locally.

```bash
npm start
```

The app will be available at `http://localhost:3000` for local testing.

## Features

1. **Document Uploads**: Upload PDF, video, and photo files as study resources.
2. **Search Functionality**: Search for keywords or specific concepts within uploaded documents.
3. **Summarization Tool**: Generate summaries of long documents for quicker review.


## Built With

- **React.js** - For building the frontend interface
- **Node.js** - Backend processing and API handling
- **Natural Language Processing** tools - For document summarization and keyword extraction

## Contributing

Contributions are welcome! Please read `CONTRIBUTING.md` for our code of conduct and submission guidelines.

## Authors

- **Devansh Alok** - Initial work - [devanshalok](https://github.com/devanshalok)
- **Joshua Sun** - Initial work - [joshuasun](https://github.com/joshuasun)
- **Ramsha Tasneem** - Initial work - [ramshatasneem](https://github.com/ramshatasneem)

## License

This project is licensed under the MIT License - see the `LICENSE.md` file for details.

## Acknowledgments

- Inspiration from student organization tools and study aids.
- Thanks to the open-source community for contributing to document management and NLP libraries.
- Gratitude to anyone who provided feedback and helped improve Study Vault’s functionality.


## Detected evidence (automated analysis)

Indexed codebase: 21 recognized source files, 702 KB.
- CSS (language) — detected in the code
- Express (technology) — detected in the code
- HTML (language) — detected in the code
- JavaScript (language) — detected in the code
- React (technology) — detected in the code
- Tailwind CSS (technology) — detected in the code
- Firebase (technology) — claimed on Devpost, not found in the code
- Next.js (technology) — claimed on Devpost, not found in the code

## Codebase structure (from repository index)

### Files (25 of 25)

```
README.md
study-vault/.gitignore
study-vault/package.json
study-vault/postcss.config.js
study-vault/public/index.html
study-vault/public/manifest.json
study-vault/public/pdf.worker.min.js
study-vault/public/robots.txt
study-vault/README.md
study-vault/src/App.css
study-vault/src/App.js
study-vault/src/App.test.js
study-vault/src/Header.js
study-vault/src/imageUtils.js
study-vault/src/index.css
study-vault/src/index.js
study-vault/src/Lightbox.js
study-vault/src/MainContent.js
study-vault/src/pdfUtils.js
study-vault/src/reportWebVitals.js
study-vault/src/server.js
study-vault/src/setupTests.js
study-vault/src/Sidebar.js
study-vault/src/UploadButton.js
study-vault/tailwind.config.js
```

### Dependencies

- study-vault/package.json: @google-cloud/speech@^6.7.0, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^13.5.0, autoprefixer@^10.4.20, axios@^1.7.7, express@^4.21.1, ffmpeg@^0.0.4, fluent-ffmpeg@^2.1.3, multer@^1.4.5-lts.1, pdfjs-dist@^2.6.347, postcss@^8.4.47, react@^18.3.1, react-dom@^18.3.1, react-icons@^5.3.0, react-scripts@5.0.1, react-toastify@^10.0.6, speech-to-text@^2.9.1, tailwindcss@^3.4.14, web-vitals@^2.1.4

### Recent commits (newest first)

- Update README.md
- Update README.md
- Connecting to backend with huggingface and gemini
- make the PDFs searchable
- photos are now displayed in teh content area and clickable
- can do mutliple searches now in a conversation
- added dynamic search functionalities and some additional logic
- skeleton of the app
- Initial commit

## Key source files (fetched from GitHub, selected and truncated for size)

### study-vault/package.json

```
{
  "name": "study-vault",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@google-cloud/speech": "^6.7.0",
    "@testing-library/jest-dom": "^5.17.0",
    "@testing-library/react": "^13.4.0",
    "@testing-library/user-event": "^13.5.0",
    "axios": "^1.7.7",
    "express": "^4.21.1",
    "ffmpeg": "^0.0.4",
    "fluent-ffmpeg": "^2.1.3",
    "multer": "^1.4.5-lts.1",
    "pdfjs-dist": "^2.6.347",
    "react": "^18.3.1",
    "react-dom": "^18.3.1",
    "react-icons": "^5.3.0",
    "react-scripts": "5.0.1",
    "react-toastify": "^10.0.6",
    "speech-to-text": "^2.9.1",
    "web-vitals": "^2.1.4"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": [
      "react-app",
      "react-app/jest"
    ]
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "devDependencies": {
    "autoprefixer": "^10.4.20",
    "postcss": "^8.4.47",
    "tailwindcss": "^3.4.14"
  }
}

```

### study-vault/src/index.js

```javascript
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';
import reportWebVitals from './reportWebVitals';
import 'react-toastify/dist/ReactToastify.css';


const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

```

### study-vault/src/server.js

```javascript
const express = require('express');
const multer = require('multer');
const ffmpeg = require('fluent-ffmpeg');
const speech = require('@google-cloud/speech');
const cors = require('cors');
const fs = require('fs');
const path = require('path');

const app = express();
app.use(cors());
app.use(express.json());

const upload = multer({ dest: 'uploads/' });

const client = new speech.SpeechClient({
  keyFilename: 'coral-velocity-439206-b7-6d21b05ffffe.json',
});

// Endpoint to upload videos
app.post('/upload-video', upload.single('video'), async (req, res) => {
  const videoFilePath = req.file.path;
  const audioFilePath = `${videoFilePath}.wav`;

  // Extract audio from video using FFmpeg
  ffmpeg(videoFilePath)
    .output(audioFilePath)
    .noVideo()
    .audioCodec('pcm_s16le')
    .audioChannels(1)
    .audioFrequency(16000)
    .on('end', async () => {
      // Transcribe audio
      const file = fs.readFileSync(audioFilePath);
      const audioBytes = file.toString('base64');

      const audio = {
        content: audioBytes,
      };

      const config = {
        encoding: 'LINEAR16',
        sampleRateHertz: 16000,
        languageCode: 'en-US',
        enableWordTimeOffsets: true,
      };

      const request = {
        audio: audio,
        config: config,
      };

      try {
        const [response] = await client.recognize(request);
        const transcription = response.results
          .map((result) => result.alternatives[0].transcript)
          .join('\n');

        const words = [];
        response.results.forEach((result) => {
          result.alternatives[0].words.forEach((wordInfo) => {
            words.push(wordInfo);
          });
        });
        console.log(words);


        // Clean up files
        fs.unlinkSync(videoFilePath);
        fs.unlinkSync(audioFilePath);

        res.json({ transcription, words });
      } catch (error) {
        console.error('Error transcribing audio:', error);
        res.status(500).json({ error: 'Error transcribing audio' });
      }
    })
    .on('error', (err) => {
      console.error('Error extracting audio:', err);
      res.status(500).json({ error: 'Error extracting audio' });
    })
    .run();
});

app.listen(4000, () => {
  console.log('Server started on port 4000');
});

```

### study-vault/src/App.js

```javascript
import React, { useState } from 'react';
import Header from './Header';
import Sidebar from './Sidebar';
import MainContent from './MainContent';
import UploadButton from './UploadButton';
import { ToastContainer, toast } from 'react-toastify';
import { extractTextFromImage } from './imageUtils';
import axios from 'axios';

const App = () => {
  const [filesInProgress, setFilesInProgress] = useState([]);
  const [conversations, setConversations] = useState([]);
  const [activeConversationId, setActiveConversationId] = useState(null);

  // Handle file uploads
const handleUpload = async (event) => {
  if (activeConversationId === null) {
    const uploadedFiles = Array.from(event.target.files);

    const filesWithData = await Promise.all(
      uploadedFiles.map(async (file) => {
        let extractedData = null;
        let preview = URL.createObjectURL(file);

        if (file.type.startsWith('image/')) {
          // Process images
          const text = await extractTextFromImage(file);
          console.log(text);
          extractedData = { text };
        } else if (file.type.startsWith('video/')) {
          // Upload video to server and get transcription
          const formData = new FormData();
          formData.append('video', file);

          try {
            const response = await axios.post(
              'http://localhost:4000/upload-video',
              formData,
              {
                headers: {
                  'Content-Type': 'multipart/form-data',
                },
              }
            );
            extractedData = response.data;
          } catch (error) {
            console.error('Error processing video:', error);
          }
        }

        return {
          name: file.name,
          type: file.type,
          size: file.size,
          lastModified: file.lastModified,
          preview: preview,
          extractedData: extractedData,
        };
      })
    );

    setFilesInProgress([...filesInProgress, ...filesWithData]);
  }
};

const handleSearch = async (query) => {
  if (query.trim() === '') {
    toast.warn('You should enter a value.', {
      position: 'top-right',
      autoClose: 3000,
    });
    return;
  }

  // Search through files in progress or active conversation
  let filesToSearch = [];
  if (activeConversationId === null) {
    filesToSearch = filesInProgress;
  } else {
    const activeConv = conversations.find(
      (conv) => conv.id === activeConversationId
    );
    filesToSearch = activeConv.files;
  }

  const searchResults = [];

  for (const file of filesToSearch) {
    if (file.type.startsWith('image/') && file.extractedData?.text) {
      // Search in image text
      const sentences = file.extractedData.text.split('\n');
      const matchedSentences = sentences.filter((sentence) =>
        sentence.toLowerCase().includes(query.toLowerCase())
      );
      if (matchedSentences.length > 0) {
        searchResults.push({
          fileName: file.name,
          type: 'image',
          matches: matchedSentences,
        });
      }
    } else if (
      file.type.startsWith('video/') &&
      file.extractedData?.transcription
    ) {
      // Search in video transcription
      const words = file.extractedData.words;
      const matchedWords = words.filter((wordInfo) =>
        wordInfo.word.toLowerCase().includes(query.toLowerCase())
      );
      if (matchedWords.length > 0) {
        searchResults.push({
          fileName: file.name,
          type: 'video',
          matches: matchedWords,
        });
      }
    }
  }

  if (activeConversationId === null) {
    // Create a new conversation
    const newConversation = {
      id: conversations.length + 1,
      queries: [query],
      results: [searchResults],
      files: filesInProgress,
    };
    setConversations([newConversation, ...conversations]);
    setActiveConversationId(newConversation.id);
    setFilesInProgress([]);
  } else {
    // Append to the existing conversation
    setConversations(
      conversations.map((conv) =>
        conv.id === activeConversationId
          ? {
              ...conv,
              queries: [...conv.queries, query],
              results: [...conv.results, searchResults],
            }
          : conv
      )
    );
  }
};

  // Handle conversation selection
  const handleConversationClick = (id) => {
    setActiveConversationId(id);
  };

  // Handle home button click
  const handleHomeClick = () => {
    setActiveConversationId(null);
    setFilesInProgress([]);
  };

  // Get the active conversation
  const activeConversation = conversations.find(
    (conv) => conv.id === activeConversationId
  );

  return (
    <div className="flex flex-col h-screen">
      <Header
        onSearch={handleSearch}
        onHomeClick={handleHomeClick}
        activeConversationId={activeConversationId}
      />
      <div className="flex flex-grow overflow-hidden">
        <Sidebar
          conversations={conversations}
          onConversationClick={handleConversationClick}
          activeConversationId={activeConversationId}
        />
        <MainContent
          files={activeConversation ? activeConversation.files : filesInProgress}
          conversation={activeConversation}
        />
      </div>
      {activeConversationId === null && <UploadButton onUpload={handleUpload} />}
      <ToastContainer />
    </div>
  );
};

export default App;

```

### study-vault/postcss.config.js

```javascript
module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
  },
}

```

### study-vault/tailwind.config.js

```javascript
// tailwind.config.js
module.exports = {
  purge: ['./src/**/*.{js,jsx,ts,tsx}', './public/index.html'],
  darkMode: false, // or 'media' or 'class'
  theme: {
    extend: {},
  },
  variants: {
    extend: {},
  },
  plugins: [],
};

// tailwind.config.js
module.exports = {
  purge: ['./src/**/*.{js,jsx,ts,tsx}', './public/index.html'],
  darkMode: false,
  theme: {
    extend: {
      keyframes: {
        'fade-in-left': {
          '0%': { opacity: '0', transform: 'translateX(-20px)' },
          '100%': { opacity: '1', transform: 'translateX(0)' },
        },
        'fade-in-down': {
          '0%': { opacity: '0', transform: 'translateY(-20px)' },
          '100%': { opacity: '1', transform: 'translateY(0)' },
        },
        'slide-in-left': {
          '0%': { transform: 'translateX(-100%)' },
          '100%': { transform: 'translateX(0)' },
        },
        'bounce-in': {
          '0%': { transform: 'scale(0.5)', opacity: '0' },
          '60%': { transform: 'scale(1.2)', opacity: '1' },
          '100%': { transform: 'scale(1)', opacity: '1' },
        },
        'fade-in-up': {
          '0%': { opacity: '0', transform: 'translateY(20px)' },
          '100%': { opacity: '1', transform: 'translateY(0)' },
        },
      },
      animation: {
        'fade-in-left': 'fade-in-left 0.5s ease-out forwards',
        'fade-in-down': 'fade-in-down 0.5s ease-out forwards',
        'slide-in-left': 'slide-in-left 0.5s ease-out forwards',
        'bounce-in': 'bounce-in 0.5s ease-out forwards',
        'fade-in-up': 'fade-in-up 0.5s ease-out forwards',
      },
    },
  },
  variants: {
    extend: {},
  },
  plugins: [],
};

```

### study-vault/src/index.css

```css
/* index.css */
@tailwind base;
@tailwind components;
@tailwind utilities;

/* Custom styles (if any) can go here */
```

### study-vault/src/setupTests.js

```javascript
// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

```

### study-vault/src/App.test.js

```javascript
import { render, screen } from '@testing-library/react';
import App from './App';

test('renders learn react link', () => {
  render(<App />);
  const linkElement = screen.getByText(/learn react/i);
  expect(linkElement).toBeInTheDocument();
});

```

### study-vault/src/reportWebVitals.js

```javascript
const reportWebVitals = onPerfEntry => {
  if (onPerfEntry && onPerfEntry instanceof Function) {
    import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => {
      getCLS(onPerfEntry);
      getFID(onPerfEntry);
      getFCP(onPerfEntry);
      getLCP(onPerfEntry);
      getTTFB(onPerfEntry);
    });
  }
};

export default reportWebVitals;

```

[9 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]