# Project export: SecondSearch This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: Cal Hacks 10.0 - Tagline: Instantly search large lecture series to answer your questions! - Devpost: https://devpost.com/software/secondsearch - GitHub: https://github.com/KanishkGar/calhacks - Result: winner (Best Use of Milvus; Best Designed Human-Computer Interaction) - Team: 2 GitHub contributor(s) — Kanishk Garg (11 commits), catherine (3 commits) ## Devpost submission (written by the team) ### Inspiration With the Berkeley enrollment time just around the corner, everyone is stressed about what classes to take. Recently, we had a conversation with one of our friends who was especially stressed about taking CS 162 next semester, with her main concern being that the course has so much content and it will be hard for her to process and digest all the information before midterms. We got the idea to create SecondSearch, where her and all other students in any class can quickly and efficiently review class material by searching through lectures directly. ### What it does SecondSearch answers any question about a course with a direct link to the lecture which explains the question. It performs a vector similarity search to determine which portion of lecture is most likely to answer your question and then displays that video. ### How we built it We built SecondSearch on the Milvus open-source vector database, using OpenAI to help with the search, then completed the product with a companion React frontend built with Chakra UI component library. We implemented the backend using FastAPI and populated the Milvus docker containers with Jupyter Notebook. ### Challenges we ran into We had trouble setting up Milvus and Docker at first, but were quickly able to find thorough documentation for the setup process. Working with React and frontend in general for the first time, we took a couple hours ramping up. It was smooth sailing after the difficult ramp up process :) ### Accomplishments we're proud of We're proud of getting a full stack product working in the short span of the hackathon: the client, server, and Milvus docker instance. ### What we learned We learned how to use Docker, FastAPI, and React, as well as the basics (struggles) of full stack development. ### What's next After creating the minimum viable product, we wanted to make the UI more friendly by using OpenAI to summarize the caption display from the video segments. However, we quickly realized that adding this change would slow the search time down from its current ~1 second to ~20 seconds. As we ran out of time to speed up this feature, we decided to temporarily remove it. However, we will be reimplementing it more efficiently as soon as possible. As for the big picture and the more distant future, currently our product works with lecture series uploaded to Youtube - we want to expand to lecture videos uploaded to other platforms, as some Berkeley classes upload recordings to bCourses, and other institutions use different platforms. After we expand the project further, some reaching goals for the far future include advertising the completed product to all university students, as lectures are often recorded and uploaded in some form. We also want to add new features on future patches such as saving previous searches, and more. ## README (from the GitHub repository) ## Inspiration With the Berkeley enrollment time just around the corner, everyone is stressed about what classes to take. Recently, we had a conversation with one of our friends who was especially stressed about taking CS 162 next semester, with her main concern being that the course has so much content and it will be hard for her to process and digest all the information before midterms. We got the idea to create SecondSearch, where her and all other students in any class can quickly and efficiently review class material by searching through lectures directly. ## What it does SecondSearch answers any question about a course with a direct link to the lecture which explains the question. It performs a vector similarity search to determine which portion of lecture is most likely to answer your question and then displays that video. ## How we built it We built SecondSearch on the Milvus open-source vector database, using OpenAI to help with the search, then completed the product with a companion React frontend built with Chakra UI component library. The backend was made using FastAPI and the Milvus docker containers were populated using Jupyter notebooks. ## Challenges we ran into We had trouble setting up Milvus and Docker at first. We also were new to React so had some difficulties getting that to work. ## Accomplishments that we're proud of We're proud of getting a full stack product working: the client, server, and Milvus docker instance. ## What we learned We learned how to use Docker, FastAPI, and React ## What's next for SecondSearch After creating the minimum viable product, we wanted to make the UI more friendly by using OpenAI to summarize the caption display from the video segments. However, we quickly realized that adding this change would slow the search time down from its current ~1 second to ~20 seconds. As we ran out of time to speed up this feature, we decided to temporarily remove it. However, we will be reimplementing it more efficiently as soon as possible. As for the big picture and the more distant future, currently our product works with lecture series uploaded to Youtube - we want to expand to lecture videos uploaded to other platforms, as some Berkeley classes upload recordings to bCourses, and other institutions use different platforms. After we expand the project further, some reaching goals for the far future include advertising the completed product to all university students, as lectures are often recorded and uploaded in some form. We also want to add new features on future patches such as saving previous searches, and more. Built With Milvus Open Source Vector Database FastAPI Python React ## Detected evidence (automated analysis) Indexed codebase: 23 recognized source files, 34 KB. - FastAPI (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - OpenAI (technology) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code ## Codebase structure (from repository index) ### Files (88 of 88) ``` .gitignore LICENSE my-app/.gitignore my-app/.prettierrc my-app/package.json my-app/public/index.html my-app/public/manifest.json my-app/public/robots.txt my-app/README.md my-app/src/App.js my-app/src/App.test.js my-app/src/Boxes.js my-app/src/ColorModeSwitcher.js my-app/src/index.js my-app/src/Logo.js my-app/src/Navbar.js my-app/src/reportWebVitals.js my-app/src/SearchBar.js my-app/src/serviceWorker.js my-app/src/setupTests.js my-app/src/test-utils.js my-app/src/Verticalstack.js README.md scrape_process/cs162_1.txt scrape_process/cs162_pickle/1.pickle scrape_process/cs162_pickle/10.pickle scrape_process/cs162_pickle/11.pickle scrape_process/cs162_pickle/12.pickle scrape_process/cs162_pickle/13.pickle scrape_process/cs162_pickle/14.pickle scrape_process/cs162_pickle/15.pickle scrape_process/cs162_pickle/16.pickle scrape_process/cs162_pickle/17.pickle scrape_process/cs162_pickle/18.pickle scrape_process/cs162_pickle/19.pickle scrape_process/cs162_pickle/2.pickle scrape_process/cs162_pickle/20.pickle scrape_process/cs162_pickle/21.pickle scrape_process/cs162_pickle/22.pickle scrape_process/cs162_pickle/23.pickle scrape_process/cs162_pickle/24.pickle scrape_process/cs162_pickle/25.pickle scrape_process/cs162_pickle/26.pickle scrape_process/cs162_pickle/3.pickle scrape_process/cs162_pickle/4.pickle scrape_process/cs162_pickle/5.pickle scrape_process/cs162_pickle/6.pickle scrape_process/cs162_pickle/7.pickle scrape_process/cs162_pickle/8.pickle scrape_process/cs162_pickle/9.pickle scrape_process/cs162/1.txt scrape_process/cs162/10.txt scrape_process/cs162/11.txt scrape_process/cs162/12.txt scrape_process/cs162/13.txt scrape_process/cs162/14.txt scrape_process/cs162/15.txt scrape_process/cs162/16.txt scrape_process/cs162/17.txt scrape_process/cs162/18.txt scrape_process/cs162/19.txt scrape_process/cs162/2.txt scrape_process/cs162/20.txt scrape_process/cs162/21.txt scrape_process/cs162/22.txt scrape_process/cs162/23.txt scrape_process/cs162/24.txt scrape_process/cs162/25.txt scrape_process/cs162/26.txt scrape_process/cs162/3.txt scrape_process/cs162/4.txt scrape_process/cs162/5.txt scrape_process/cs162/6.5.txt scrape_process/cs162/6.txt scrape_process/cs162/7.txt scrape_process/cs162/8.txt scrape_process/cs162/9.txt scrape_process/first_video.pkl scrape_process/imdb_search_milvus_client.ipynb scrape_process/process_text.ipynb server/requirements.txt server/scripts/activate_venv.sh server/scripts/run.sh server/scripts/setup_venv.sh server/src/api.py server/src/util/milvus_model.py server/src/util/responsemodel.py server/src/util/util_function.py ``` ### Dependencies - my-app/package.json: @chakra-ui/icons@^2.1.1, @chakra-ui/react@^2.8.1, @emotion/react@^11.11.1, @emotion/styled@^11.11.0, @testing-library/jest-dom@^5.17.0, @testing-library/react@^13.4.0, @testing-library/user-event@^14.5.1, framer-motion@^6.5.1, react@^18.2.0, react-dom@^18.2.0, react-icons@^3.11.0, react-scripts@5.0.1, web-vitals@^2.1.4 - server/requirements.txt: aiohttp@==3.8.6, aiosignal@==1.3.1, annotated-types@==0.6.0, anyio@==3.7.1, async-timeout@==4.0.3, attrs@==23.1.0, certifi@==2023.7.22, charset-normalizer@==3.3.1, click@==8.1.7, environs@==9.5.0, exceptiongroup@==1.1.3, fastapi@==0.104.0, frozenlist@==1.4.0, grpcio@==1.58.0, h11@==0.14.0, idna@==3.4, marshmallow@==3.20.1, milvus@==2.3.0, minio@==7.1.17, multidict@==6.0.4, numpy@==1.26.1, openai@==0.28.1, packaging@==23.2, pandas@==2.1.2, protobuf@==4.24.4, pydantic@==2.4.2, pydantic_core@==2.10.1, pymilvus@==2.3.2, python-dateutil@==2.8.2, python-dotenv@==1.0.0, pytz@==2023.3.post1, requests@==2.31.0, six@==1.16.0, sniffio@==1.3.0, starlette@==0.27.0, tqdm@==4.66.1, typing_extensions@==4.8.0, tzdata@==2023.3, ujson@==5.8.0, urllib3@==2.0.7, uvicorn@==0.23.2, yarl@==1.9.2 ### Recent commits (newest first) - Update README.md - Merge branch 'main' of https://github.com/KanishkGar/calhacks - added readme - delete unecssary frontends - fixed null issue - fix - pickles + finished - pretty much final frontend - initial app - server done - req - backend kinda working lol - scraping and processing code - rename - gitignore ipynb pkl - captions - initial server fastapi commit - Initial commit ## Key source files (fetched from GitHub, selected and truncated for size) ### server/requirements.txt ``` aiohttp==3.8.6 aiosignal==1.3.1 annotated-types==0.6.0 anyio==3.7.1 async-timeout==4.0.3 attrs==23.1.0 certifi==2023.7.22 charset-normalizer==3.3.1 click==8.1.7 environs==9.5.0 exceptiongroup==1.1.3 fastapi==0.104.0 frozenlist==1.4.0 grpcio==1.58.0 h11==0.14.0 idna==3.4 marshmallow==3.20.1 milvus==2.3.0 minio==7.1.17 multidict==6.0.4 numpy==1.26.1 openai==0.28.1 packaging==23.2 pandas==2.1.2 protobuf==4.24.4 pydantic==2.4.2 pydantic_core==2.10.1 pymilvus==2.3.2 python-dateutil==2.8.2 python-dotenv==1.0.0 pytz==2023.3.post1 requests==2.31.0 six==1.16.0 sniffio==1.3.0 starlette==0.27.0 tqdm==4.66.1 typing_extensions==4.8.0 tzdata==2023.3 ujson==5.8.0 urllib3==2.0.7 uvicorn==0.23.2 yarl==1.9.2 ``` ### my-app/package.json ``` { "name": "my-app", "version": "0.1.0", "private": true, "dependencies": { "@chakra-ui/icons": "^2.1.1", "@chakra-ui/react": "^2.8.1", "@emotion/react": "^11.11.1", "@emotion/styled": "^11.11.0", "@testing-library/jest-dom": "^5.17.0", "@testing-library/react": "^13.4.0", "@testing-library/user-event": "^14.5.1", "framer-motion": "^6.5.1", "react": "^18.2.0", "react-dom": "^18.2.0", "react-icons": "^3.11.0", "react-scripts": "5.0.1", "web-vitals": "^2.1.4" }, "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": "react-app" }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] } } ``` ### my-app/src/App.js ```javascript import React from 'react'; import { ChakraProvider, Box, Text, Link, VStack, Code, Grid, theme, Center, } from '@chakra-ui/react'; import { ColorModeSwitcher } from './ColorModeSwitcher'; import { Logo } from './Logo'; import Navbar from './Navbar'; import Verticalstack from './Verticalstack'; function App() { return ( ); } export default App; ``` ### my-app/src/index.js ```javascript import { ColorModeScript } from '@chakra-ui/react'; import React, { StrictMode } from 'react'; import * as ReactDOM from 'react-dom/client'; import App from './App'; import reportWebVitals from './reportWebVitals'; import * as serviceWorker from './serviceWorker'; const container = document.getElementById('root'); const root = ReactDOM.createRoot(container); root.render( ); // If you want your app to work offline and load faster, you can change // unregister() to register() below. Note this comes with some pitfalls. // Learn more about service workers: https://cra.link/PWA serviceWorker.unregister(); // If you want to start measuring performance in your app, pass a function // to log results (for example: reportWebVitals(console.log)) // or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals reportWebVitals(); ``` ### server/scripts/run.sh ```shell uvicorn src.api:app --reload --lifespan on --host 0.0.0.0 --port 8080 ``` ### my-app/src/setupTests.js ```javascript // jest-dom adds custom jest matchers for asserting on DOM nodes. // allows you to do things like: // expect(element).toHaveTextContent(/react/i) // learn more: https://github.com/testing-library/jest-dom import '@testing-library/jest-dom'; ``` ### my-app/src/App.test.js ```javascript import React from 'react'; import { screen } from '@testing-library/react'; import { render } from './test-utils'; import App from './App'; test('renders learn react link', () => { render(); const linkElement = screen.getByText(/learn chakra/i); expect(linkElement).toBeInTheDocument(); }); ``` ### my-app/src/reportWebVitals.js ```javascript const reportWebVitals = onPerfEntry => { if (onPerfEntry && onPerfEntry instanceof Function) { import('web-vitals').then(({ getCLS, getFID, getFCP, getLCP, getTTFB }) => { getCLS(onPerfEntry); getFID(onPerfEntry); getFCP(onPerfEntry); getLCP(onPerfEntry); getTTFB(onPerfEntry); }); } }; export default reportWebVitals; ``` ### my-app/src/test-utils.js ```javascript import React from 'react'; import { render } from '@testing-library/react'; import { ChakraProvider, theme } from '@chakra-ui/react'; const AllProviders = ({ children }) => ( {children} ); const customRender = (ui, options) => render(ui, { wrapper: AllProviders, ...options }); export { customRender as render }; ``` ### my-app/src/Logo.js ```javascript import React from 'react'; import { Image, keyframes, usePrefersReducedMotion } from '@chakra-ui/react'; import logo from './logo.svg'; const spin = keyframes` from { transform: rotate(0deg); } to { transform: rotate(360deg); } `; export const Logo = props => { const prefersReducedMotion = usePrefersReducedMotion(); const animation = prefersReducedMotion ? undefined : `${spin} infinite 20s linear`; return ; }; ``` [13 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]