# Project export: PeachDeck This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: PeachDeck expands PowerPoint slides into AI-generated video presentations so you never have to worry about missing lectures or how to teach lectures ever again. - Devpost: https://devpost.com/software/peachdeck - GitHub: https://github.com/ekang7/peachdeck - Demo: https://www.figma.com/file/cQwr93mvN2v6PoRLzj3Q2I/PeachDeck?type=design&node-id=1%3A1411&mode=design&t=Ee5PTDkFNN7sUxyk-1 - Video: https://www.youtube.com/embed/-kpCRH6PlcU?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Team: 2 GitHub contributor(s) — Basil Hijaz (5 commits), Edward Kang (1 commits) ## Devpost submission (written by the team) ### Overview What it does PeachDeck turns PowerPoint slides into AI-generated video presentations. It doesn't just summarize the slides, but it uses RAG LLM model to expand upon the lecture and teach the nitty gritty of that professors didn't mention in only a matter of minutes. It can be applied for consultants, training videos, teachers, and students. ### Inspiration What happens when you miss a class and all you have is a deck of lecture slides filled with bullet point content and no context? One of our members often missed classes because of illness/other commitments, and they fell behind because lecture slides often provide insufficient context. What if there was a way for them to catch up on lectures easily? Using RAG LLM, however, we can turn disjointed lecture slides into a cohesive video presentation with immaculate generative voiceovers, allowing students to pick up missed concepts in a short amount of time. But our AI-powered journey doesn't stop there. We're not just filling gaps; we're amplifying the learning experience. Instructors often can't dive deep into every concept in class, leaving students curious about what wasn't covered. Our AI agent steps in, providing in-depth explanations for concepts left untouched in class. It's not just about catching up; it's about delving into the intricacies of each subject. The use case is not confined to schools alone – it can help with any kind of training and educational videos. It reduces the time it takes for instructors to prepare online courses, and also eliminates the need for extensive manual video production, making corporate training more budget-friendly. We hope to democratize access to high-quality content, reaching learners around the world and contributing to a more equitable education landscape. ### How we built it Use Bun to create the React app and install all needed packages Used PredictionGuard AI from Intel to create a safe RAG LLM model (AI Agent) for university textbooks Used Python for data cleaning We found open-source textbooks with free for commercial use licenses which we then embedded as vectors with multimodal CLIP We used the open-source en_core_web_md model from spaCy to classify what subject a lecture belonged to and thus indexed to the appropriate vector embedding space Used ElevenLabs for audio generation with multiple generative AI voices Used React for the frontend Used Figma for prototyping: https://www.figma.com/file/TPEmQIIjMAyELI5sIkzbeP/Trees!?type=design&node-id=93%3A26453&mode=design&t=ruoBvc9hY4f9oCzR-1 Used Flask for the backend server Used Spire to convert pptx slides to images and add audio over them ### Challenges we ran into We ran into many challenges when merging our front-end and back-end programs together, as the merge involved combining a variety of difficult conversions, such as extracting text from powerpoints, integrating text with audio, and converting audio to video. The last one took an especially long time to resolve as generating video files are often quite tedious. Asking prompts - after 4 hours of playing around, we discovered that LLMs did better with less information than with more counterintuitively ### Accomplishments we're proud of We are especially proud of the level of the pipeline we've managed to execute, using a plethora of different algorithms to make something extraordinarily useful for the education industry. End-to-end design of the web app on Figma after many iterations; we want to create something that is aesthetically pleasing and intuitive to use I have never built a RAG LLM model before and was surprised to learn how cool and fun it was, especially with mentors from Intel! ### What we learned Getting advice from experts in the field of what you're looking into is extremely helpful. By talking to experienced engineers and thinkers early on in our journey, we were able to dream big and accomplish tasks at a more monumental level. ### What's next for We hope that the product would evolve into something that allows users to edit the AI-generated content and customize it more directly. Users can review generated scripts from slides first, and regenerate if they are not satisfied. They can also edit the video directly within the platform, and use the chatbot for further assistance. Ability to add AI-generated avatars (we already have voices and the ability to change between them but do not have a clear way to do this on our current interface). In the future, we also want to add translations, so it enables people who are not fluent in English learn from slides in their native language! We hope that one day everyone can attend any lecture in the world! ## README (from the GitHub repository) ### What it does PeachDeck turns PowerPoint slides into AI-generated video presentations. It doesn't just summarize the slides, but it uses RAG LLM model to expand upon the lecture and teach the nitty gritty of that professors didn't mention in only a matter of minutes. It can be applied for consultants, training videos, teachers, and students. ### Inspiration What happens when you miss a class and all you have is a deck of lecture slides filled with bullet point content and no context? One of our members often missed classes because of illness/other commitments, and they fell behind because lecture slides often provide insufficient context. What if there was a way for them to catch up on lectures easily? Using RAG LLM, however, we can turn disjointed lecture slides into a cohesive video presentation with immaculate generative voiceovers, allowing students to pick up missed concepts in a short amount of time. But our AI-powered journey doesn't stop there. We're not just filling gaps; we're amplifying the learning experience. Instructors often can't dive deep into every concept in class, leaving students curious about what wasn't covered. Our AI agent steps in, providing in-depth explanations for concepts left untouched in class. It's not just about catching up; it's about delving into the intricacies of each subject. The use case is not confined to schools alone – it can help with any kind of training and educational videos. It reduces the time it takes for instructors to prepare online courses, and also eliminates the need for extensive manual video production, making corporate training more budget-friendly. We hope to democratize access to high-quality content, reaching learners around the world and contributing to a more equitable education landscape. ### How we built it Use Bun to create the React app and install all needed packages Used PredictionGuard AI from Intel to create a safe RAG LLM model (AI Agent) for university textbooks Used Python for data cleaning We found open-source textbooks with free for commercial use licenses which we then embedded as vectors with multimodal CLIP We used the open-source en_core_web_md model from spaCy to classify what subject a lecture belonged to and thus indexed to the appropriate vector embedding space Used ElevenLabs for audio generation with multiple generative AI voices Used React for the frontend Used Figma for prototyping: https://www.figma.com/file/TPEmQIIjMAyELI5sIkzbeP/Trees!?type=design&node-id=93%3A26453&mode=design&t=ruoBvc9hY4f9oCzR-1 Used Flask for the backend server Used Spire to convert pptx slides to images and add audio over them Challenges we ran into We ran into many challenges when merging our front-end and back-end programs together, as the merge involved combining a variety of difficult conversions, such as extracting text from powerpoints, integrating text with audio, and converting audio to video. The last one took an especially long time to resolve as generating video files are often quite tedious. Asking prompts - after 4 hours of playing around, we discovered that LLMs did better with less information than with more counterintuitively ### Accomplishments that we're proud of We are especially proud of the level of the pipeline we've managed to execute, using a plethora of different algorithms to make something extraordinarily useful for the education industry. End-to-end design of the web app on Figma after many iterations; we want to create something that is aesthetically pleasing and intuitive to use I have never built a RAG LLM model before and was surprised to learn how cool and fun it was, especially with mentors from Intel! ### What we learned Getting advice from experts in the field of what you're looking into is extremely helpful. By talking to experienced engineers and thinkers early on in our journey, we were able to dream big and accomplish tasks at a more monumental level. ### What's next for We hope that the product would evolve into something that allows users to edit the AI-generated content and customize it more directly. Users can review generated scripts from slides first, and regenerate if they are not satisfied. They can also edit the video directly within the platform, and use the chatbot for further assistance. Ability to add AI-generated avatars (we already have voices and the ability to change between them but do not have a clear way to do this on our current interface). In the future, we also want to add translations, so it enables people who are not fluent in English learn from slides in their native language! We hope that one day everyone can attend any lecture in the world! ### Built With agent ai api bun css elevenlabs figma flask html intel javascript langchain llm neurachat numpy open-source pandas python rag react spacey spire ## Detected evidence (automated analysis) Indexed codebase: 10 recognized source files, 21 KB. - CSS (language) — detected in the code - Firebase (technology) — detected in the code - Flask (technology) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - LangChain (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (120 of 1538) ``` .eslintrc.cjs .gitignore .lancedb/bio.lance/_latest.manifest .lancedb/bio.lance/_transactions/0-321866d2-655c-4b2a-ab73-9e9040d31f5a.txn .lancedb/bio.lance/_transactions/1-8fc8ff8d-6c10-4a07-bc7c-3bc34be6850b.txn .lancedb/bio.lance/_transactions/10-b0fed4d8-e6c0-498c-b3df-ca96886ae232.txn .lancedb/bio.lance/_transactions/100-95f67c80-4a53-4beb-b222-05619fffae5b.txn .lancedb/bio.lance/_transactions/101-4bad5f4e-5c57-44c9-a638-00b28cf6cc37.txn .lancedb/bio.lance/_transactions/102-6e85f410-f1c6-4991-85de-e1a50b814d7b.txn .lancedb/bio.lance/_transactions/103-d2282612-a961-4a1c-9937-ec32b52d0706.txn .lancedb/bio.lance/_transactions/104-e07327bd-56a0-4b00-8418-ffe9d0e1d708.txn .lancedb/bio.lance/_transactions/105-a4dd260e-0524-4a8d-b348-e80182bd78dd.txn .lancedb/bio.lance/_transactions/106-9f8b53ef-ea6b-45bd-b031-a7a3fc3c7a1a.txn .lancedb/bio.lance/_transactions/107-06aea9e1-9829-4310-94c4-0f41108c007b.txn .lancedb/bio.lance/_transactions/108-4078e2de-7672-4b1a-9897-4863da016e97.txn .lancedb/bio.lance/_transactions/109-d83ede19-bbdc-46c3-b57f-80cc1f1cdf90.txn .lancedb/bio.lance/_transactions/11-ada6c7b1-6fb9-441d-854d-71fd400adafb.txn .lancedb/bio.lance/_transactions/110-65d9bdbf-72c0-4091-bf55-8b336dbbae28.txn .lancedb/bio.lance/_transactions/111-ad1ef89e-a8c7-43d8-bc81-bc80fc215d9b.txn .lancedb/bio.lance/_transactions/112-d82a7782-8ead-458e-aa27-da5fe0087b22.txn .lancedb/bio.lance/_transactions/113-e88b167b-eec8-4624-9d45-53a819493fe6.txn .lancedb/bio.lance/_transactions/114-7ae8da5e-70e4-4343-8825-b1bc6018c33d.txn .lancedb/bio.lance/_transactions/115-1c3034f7-68db-414c-bbd8-ed1aafebcb02.txn .lancedb/bio.lance/_transactions/116-ab2a8d67-fbdb-4f4b-8f5d-ece9b06ca4a8.txn .lancedb/bio.lance/_transactions/117-084de562-1674-47a7-a2f0-0b098d692e59.txn .lancedb/bio.lance/_transactions/118-ec54b526-74d4-4cad-bc1e-90f27ffaa774.txn .lancedb/bio.lance/_transactions/119-8e41f409-57d7-427c-85b2-7087a59ef034.txn .lancedb/bio.lance/_transactions/12-3446d309-f3c0-44f0-9b15-4634322e1147.txn .lancedb/bio.lance/_transactions/120-a121a9a4-f5cd-4958-9832-0344a9ac14a4.txn .lancedb/bio.lance/_transactions/121-ed01c23b-dac6-4e53-97e0-91b556838fbd.txn .lancedb/bio.lance/_transactions/122-16d838e7-bfa5-4004-b698-7e86b791c52d.txn .lancedb/bio.lance/_transactions/123-22662c4f-af72-4103-b26e-812d339371fd.txn .lancedb/bio.lance/_transactions/124-36770722-9c1b-4588-b0dc-674e1f703874.txn .lancedb/bio.lance/_transactions/125-b409d5b0-a175-4fa9-9b23-7598139ad41e.txn .lancedb/bio.lance/_transactions/126-c4141961-3f56-4a5e-880a-457b0b195bc7.txn .lancedb/bio.lance/_transactions/127-310c3d67-ec53-4cc0-8099-714a5d0b28e5.txn .lancedb/bio.lance/_transactions/128-65445705-8dd0-47ec-b3f4-687e830f5723.txn .lancedb/bio.lance/_transactions/129-f3baf2ab-9cdc-42c1-b25e-e70ad5cb590e.txn .lancedb/bio.lance/_transactions/13-2b18cc2f-e04e-47a7-b376-09f4e231b748.txn .lancedb/bio.lance/_transactions/130-2ed1c002-732b-4b4d-9170-13fae61e6a53.txn .lancedb/bio.lance/_transactions/131-cefe7a73-8300-4d76-9945-a391c9d94b71.txn .lancedb/bio.lance/_transactions/132-f23ff87f-dfa7-4e64-a1d6-b5837fdfc1af.txn .lancedb/bio.lance/_transactions/133-0bfac01d-98af-4ad1-86c1-9bb4ad118392.txn .lancedb/bio.lance/_transactions/134-33ebd5b1-af4a-4d00-abde-1ddf9e5513cf.txn .lancedb/bio.lance/_transactions/135-99094ad3-e7d7-4781-b868-4ce0fd0b7b98.txn .lancedb/bio.lance/_transactions/136-68daca7e-40e5-430e-85ea-c098b641f46f.txn .lancedb/bio.lance/_transactions/137-41bab124-9b5c-45c7-bfdf-c17edcb0cbaf.txn .lancedb/bio.lance/_transactions/138-69d029fe-1adf-4646-98ad-a7a2164554e9.txn .lancedb/bio.lance/_transactions/139-d47413f2-a65f-44a8-b525-898ce2ec0d93.txn .lancedb/bio.lance/_transactions/14-4df851c0-3b40-46b0-b0f4-5262da4b1539.txn .lancedb/bio.lance/_transactions/140-2ca99549-161f-4e40-b560-9047f3902b87.txn .lancedb/bio.lance/_transactions/141-4e70c715-fb5f-40e6-9097-c747710ae034.txn .lancedb/bio.lance/_transactions/142-96dd4e8d-b644-430a-bd1e-c2deb6329339.txn .lancedb/bio.lance/_transactions/143-4a0d2e5b-b185-42d7-b2c0-a9427616ccc9.txn .lancedb/bio.lance/_transactions/144-c1bda79c-3a7c-4126-8afb-69eecaf7ad94.txn .lancedb/bio.lance/_transactions/145-3523f9a5-e0e3-4ea6-927e-3e5dae770c98.txn .lancedb/bio.lance/_transactions/146-420bf69e-ccce-404e-a113-306b5b470e68.txn .lancedb/bio.lance/_transactions/147-c6cf7243-136e-4a1a-8eb7-904a30847e32.txn .lancedb/bio.lance/_transactions/148-093b2f63-cb5e-4943-b4e0-8c903eb9c8a1.txn .lancedb/bio.lance/_transactions/149-4640bd88-41be-45f6-a8fc-3a74d2e80782.txn .lancedb/bio.lance/_transactions/15-1c3035cb-5123-43e6-bfac-f7f8497415ed.txn .lancedb/bio.lance/_transactions/150-079787d5-e459-42a2-9560-17ad2bfec437.txn .lancedb/bio.lance/_transactions/151-f1d7247e-18e6-4de9-bb1e-a46441bf084a.txn .lancedb/bio.lance/_transactions/152-e1b41043-8a72-404b-b1e1-62a6bd786aea.txn .lancedb/bio.lance/_transactions/153-1e1a05cb-5f2c-4407-bb96-896a8d7ba0af.txn .lancedb/bio.lance/_transactions/154-6bf20bab-b010-4373-9783-be4d1764bcbd.txn .lancedb/bio.lance/_transactions/155-7f5eaf76-1dba-4ec7-b55b-9f815bed6321.txn .lancedb/bio.lance/_transactions/156-eabc932b-bdf1-46ab-958b-ad22b4383222.txn .lancedb/bio.lance/_transactions/157-f98c6355-36be-4821-aad8-3c96eadddf6e.txn .lancedb/bio.lance/_transactions/158-823317ac-e78a-4488-9e3b-dd79584baf2c.txn .lancedb/bio.lance/_transactions/159-38f5a938-1912-42d0-b52e-722966357ece.txn .lancedb/bio.lance/_transactions/16-baac1a68-7479-4e58-9feb-d0b4fd45d56a.txn .lancedb/bio.lance/_transactions/160-3bfc4ffa-115a-4415-af53-d0ca541134e3.txn .lancedb/bio.lance/_transactions/161-3df10f80-9f07-4ab2-a254-c0d9766fb65c.txn .lancedb/bio.lance/_transactions/162-4df59773-8b79-42e3-8e61-5e684567bf41.txn .lancedb/bio.lance/_transactions/163-42201c68-b5e2-43ce-9277-8e638f6b06de.txn .lancedb/bio.lance/_transactions/164-1964f7a9-50d5-4c35-8d0b-dd83a674af33.txn .lancedb/bio.lance/_transactions/165-ea6ae158-31ad-47ef-b8e7-3985e85f266a.txn .lancedb/bio.lance/_transactions/166-d9720d80-de35-4fea-a7cc-6b10fdc9fed9.txn .lancedb/bio.lance/_transactions/167-b2b7000a-4f9b-4187-ae59-a3ded66d7fdd.txn .lancedb/bio.lance/_transactions/168-8e5e5995-5e28-4ecf-b2de-a00db2790f58.txn .lancedb/bio.lance/_transactions/169-96763010-b02e-43df-8c5a-b508925ed26f.txn .lancedb/bio.lance/_transactions/17-5f61c715-d70d-49ac-a569-1febedca7065.txn .lancedb/bio.lance/_transactions/170-46a5984e-c874-498b-9513-5ecbfdd08a38.txn .lancedb/bio.lance/_transactions/171-4b548424-8daa-4497-9d4f-7a9558137b6f.txn .lancedb/bio.lance/_transactions/172-98c9fd06-4209-495f-b87f-b250dd5f605d.txn .lancedb/bio.lance/_transactions/173-acbaafe6-6c76-4e11-bcc1-ecd5da255f35.txn .lancedb/bio.lance/_transactions/174-c9064957-7abb-47a1-af2e-80c81738dc47.txn .lancedb/bio.lance/_transactions/175-6e174855-ed21-44e2-b81f-340242e7d174.txn .lancedb/bio.lance/_transactions/176-1bd0cb25-c985-4809-9adc-e3f85879179e.txn .lancedb/bio.lance/_transactions/177-1f8bd6b7-5135-449d-aae9-00e29dda4789.txn .lancedb/bio.lance/_transactions/178-5a561a3a-8c72-45a0-b4d5-78bc098a78a6.txn .lancedb/bio.lance/_transactions/179-06d763d8-bf71-4da4-9d48-dd07423e8002.txn .lancedb/bio.lance/_transactions/18-a67f580b-093f-4e2a-bc45-1a33973db900.txn .lancedb/bio.lance/_transactions/180-29c543ed-1307-40d6-bc89-cd278f07c98d.txn .lancedb/bio.lance/_transactions/181-9a4877e1-32e1-4dd2-a499-7bd78c603e75.txn .lancedb/bio.lance/_transactions/182-a12f8409-1fc6-4e0a-a657-8efbcd815815.txn .lancedb/bio.lance/_transactions/183-2c767bbe-ee89-4297-b04c-c4cea078d051.txn .lancedb/bio.lance/_transactions/184-e495270f-9ba3-4410-8b29-50e9ee724bfa.txn .lancedb/bio.lance/_transactions/185-14127831-dccd-43f1-ab83-18127af5fd3c.txn .lancedb/bio.lance/_transactions/186-b2a1741e-b30b-40b9-b19e-56121d7efe8a.txn .lancedb/bio.lance/_transactions/187-9b24a1f7-b867-409a-b65e-c31774162d55.txn .lancedb/bio.lance/_transactions/188-f11afc26-fd04-4e26-b167-788832ef9588.txn .lancedb/bio.lance/_transactions/189-01824ca7-622f-44ec-8bd7-0620f37fd5cd.txn .lancedb/bio.lance/_transactions/19-ca1b7d5e-ce50-4bd2-b2a3-5ebb922406ab.txn .lancedb/bio.lance/_transactions/190-49c34e00-9a15-4bf8-9e4d-757dbf9f9299.txn .lancedb/bio.lance/_transactions/191-8272d7df-e21a-44bc-a56a-d74998aebd3e.txn .lancedb/bio.lance/_transactions/192-97edb5ed-ee4b-4d83-86e2-0e1d223644dd.txn .lancedb/bio.lance/_transactions/193-120e3414-97f3-406e-93e8-2f07c194a9e8.txn .lancedb/bio.lance/_transactions/194-d9ee2339-b6e8-421a-b508-2d5edf3ef060.txn .lancedb/bio.lance/_transactions/195-3898185d-41f1-4e15-b437-a5ec2e41693c.txn .lancedb/bio.lance/_transactions/196-a7385755-4328-4c23-bae9-27249e0f2e34.txn .lancedb/bio.lance/_transactions/197-4b342ee1-d0bc-40fa-b92f-2378830fce7c.txn .lancedb/bio.lance/_transactions/198-8184d800-5286-4dec-88a8-8c11e93b4b4a.txn .lancedb/bio.lance/_transactions/199-a8590ee4-4f46-4ae8-8faf-05b6edb199da.txn .lancedb/bio.lance/_transactions/2-1b385ba7-7f54-4bd0-b4e2-f92288577b53.txn .lancedb/bio.lance/_transactions/20-e1916c63-2509-4be4-9d5e-ffffc61ce4f9.txn .lancedb/bio.lance/_transactions/200-bd0d859e-a141-4820-b7c6-74a6546b6fab.txn .lancedb/bio.lance/_transactions/201-fe01c168-0db5-49c9-9e6b-56d9197ef836.txn .lancedb/bio.lance/_transactions/202-c9387585-686f-45e8-85b1-acf0cd8ca687.txn [1418 more files omitted for size] ``` ### Dependencies - package.json: @types/react@^18.2.55, @types/react-dom@^18.2.19, @vitejs/plugin-react@^4.2.1, axios@^1.6.7, dev@^0.1.3, eslint@^8.56.0, eslint-plugin-react@^7.33.2, eslint-plugin-react-hooks@^4.6.0, eslint-plugin-react-refresh@^0.4.5, ffmpeg@^0.0.4, file-saver@^2.0.5, firebase@^10.8.0, flask@^0.2.10, git@^0.1.5, jszip@^3.10.1, pptxgenjs@^3.12.0, pspdfkit@^2024.1.3, python@^0.0.4, react@^18.2.0, react-dom@^18.2.0, vite@^5.1.3, xml-js@^1.6.11 ### Recent commits (newest first) - Update README.md - design update - added jupyters - Deleted pptx after video creation - Added conversion from audio and pptx files to video - Adding a few comments for data - Client Sending pptx file to server side - Created basic react-app to extract text out of pptx files and send this text to Flask server ## Key source files (fetched from GitHub, selected and truncated for size) ### package.json ``` { "name": "treehacks", "private": true, "version": "0.0.0", "type": "module", "proxy": "http://localhost:5000/", "scripts": { "dev": "vite", "build": "vite build", "lint": "eslint . --ext js,jsx --report-unused-disable-directives --max-warnings 0", "preview": "vite preview" }, "dependencies": { "axios": "^1.6.7", "dev": "^0.1.3", "ffmpeg": "^0.0.4", "file-saver": "^2.0.5", "firebase": "^10.8.0", "flask": "^0.2.10", "git": "^0.1.5", "jszip": "^3.10.1", "pptxgenjs": "^3.12.0", "pspdfkit": "^2024.1.3", "python": "^0.0.4", "react": "^18.2.0", "react-dom": "^18.2.0", "xml-js": "^1.6.11" }, "devDependencies": { "@types/react": "^18.2.55", "@types/react-dom": "^18.2.19", "@vitejs/plugin-react": "^4.2.1", "eslint": "^8.56.0", "eslint-plugin-react": "^7.33.2", "eslint-plugin-react-hooks": "^4.6.0", "eslint-plugin-react-refresh": "^0.4.5", "vite": "^5.1.3" } } ``` ### src/main.jsx ```javascript import React from 'react' import ReactDOM from 'react-dom/client' import App from './App.jsx' import './index.css' ReactDOM.createRoot(document.getElementById('root')).render( , ) ``` ### src/App.jsx ```javascript import React from 'react'; import PptxReader from './components/PptxReader'; import './App.css'; function App() { return (

PeachDeck

); } export default App; ``` ### backend/server.py ```python from flask import Flask, request, jsonify from flask_cors import CORS from pptx import Presentation as Pre import os from moviepy.editor import * from spire.presentation.common import * from spire.presentation import * import spacy import os import urllib.request import html2text import predictionguard as pg from langchain import PromptTemplate, FewShotPromptTemplate from langchain.text_splitter import CharacterTextSplitter from sentence_transformers import SentenceTransformer import numpy as np from getpass import getpass import lancedb from lancedb.embeddings import with_embeddings import pandas as pd from elevenlabs import generate, play, set_api_key set_api_key("5021690210e628f74d2dede4a39b6e9a") # Eleven Labs API Key # Given an input (string), index, and slide number, returns the mp3 file to the generated audio for the string with the corresponding avatar. def generateAudio(input, i, slide_no): names = ["Voice1", "Voice2", "Voice3", "Voice4", "Voice5", "Voice6"] # Generate the bytes for the output audio audio = generate( text = input, voice = names[i] ) # play(audio, notebook=True) # The path where you want to save the mp3 file file_path = "slide" + str(slide_no) + ".mp3" with open(file_path, "wb") as file: file.write(audio) return file_path #NICE # Given an array of inputs and an index selecting the avatar, generates an array of mp3 files def generateAllFiles(inputs, i): slide_no = 1 files = [] for input in inputs: fpath = generateAudio(input, i, slide_no) files.append(fpath) slide_no += 1 return files # os.environ["FFMPEG_PATH"] = './venv/lib/python3.10/site-packages/ffmpeg' from pydub import AudioSegment app = Flask(__name__) CORS(app) CORS(app, resources={r"/api/endpoint/upload": {"origins": "http://127.0.0.1:5173"}}) # Function to read audio files def read_audio(audio_file): return AudioSegment.from_file(audio_file) # Function to read PowerPoint file def read_pptx(pptx_file): return Pre(pptx_file) def convert_pptx_to_images(pptx_file): prs = Presentation() prs.LoadFromFile(pptx_file) for i, slide in enumerate(prs.Slides): # Specify the output file name fileName = f"slide_{i}.png" # Save each slide as a PNG image image = slide.SaveAsImage() image.Save(fileName) image.Dispose() prs.Dispose() # Function to create video with audio narration def create_video(pptx_file, audio_files, output_video): prs = read_pptx(pptx_file) clips = [] for i, slide in enumerate(prs.slides): audio_file = audio_files[i] audio = read_audio(audio_file) slide_image = f"slide_{i}.png" # assuming you want to use images of slides # You need to implement a function to convert slides to images # Create a clip for each slide with its corresponding audio slide_clip = ImageClip(slide_image).set_duration(audio.duration_seconds) audio_clip = AudioFileClip(audio_file) audio_clip = audio_clip.set_duration(audio.duration_seconds) slide_clip = slide_clip.set_audio(audio_clip) clips.append(slide_clip) # Concatenate all clips to form the video final_clip = concatenate_videoclips(clips) final_clip.write_videofile(output_video, fps=24) def delete_images(): directory = os.getcwd() for filename in os.listdir(directory): if filename.endswith(".png") or filename.endswith(".jpg") or filename.endswith(".pptx"): # Specify the file extensions you want to delete filepath = os.path.join(directory, filename) os.remove(filepath) # Load the medium model with word vectors nlp = spacy.load("en_core_web_md") # The two reference terms biology = nlp("Biology") history = nlp("History") # Function to determine similarity def find_closest_topic(text): # Convert the input text to a spaCy document doc = nlp(text) # Calculate similarity with 'Biology' and 'History' similarity_to_biology = doc.similarity(biology) similarity_to_history = doc.similarity(history) # Determine which similarity is greater if similarity_to_biology > similarity_to_history: return "Biology", similarity_to_biology else: return "History", similarity_to_history def embed_batch(batch): return [model.encode(sentence[:80]) for sentence in batch] def embed(sentence): return model.encode(sentence) # LanceDB setup dir_name = ".lancedb" if not os.path.exists(dir_name): os.mkdir(dir_name) uri = ".lancedb" db = lancedb.connect(uri) # Embeddings setup name="clip-ViT-B-32" model = SentenceTransformer(name) pg_access_token = 'q1VuOjnffJ3NO2oFN8Q9m8vghYc84ld13jaqdF7E' os.environ['PREDICTIONGUARD_TOKEN'] = pg_access_token # Now let's augment our Q&A prompt with this external knowledge on-the-fly!!! template = """### System: Use the below input Context from the user to create a presentation script that is less than 50 tokens about the Subject from the user. If you cannot answer the question, respond with "Sorry, I can't find an answer, but you might try looking in the following resource." ### User: Context: {context} Subject: {subject} ### Assistant: """ qa_prompt = PromptTemplate( input_variables=["context", "subject"], template=template, ) history_presentation = False biology_presentation = False def rag_answer(subject): global history_presentation, biology_presentation # opening up the data tables # Example usage if not history_presentation and not biology_presentation: input_text = subject closest_topic, similarity_score = find_closest_topic(input_text) if (closest_topic == "history"): history_presentation = True table = db.open_table("history") else: biology_presentation = True table = db.open_table("bio") elif history_presentation: table = db.open_table("history") elif biology_presentation: table = db.open_table("bio") # Search the for relevant context [truncated — 2349 more characters] ``` ### vite.config.js ```javascript import { defineConfig } from 'vite' import react from '@vitejs/plugin-react' // https://vitejs.dev/config/ export default defineConfig({ plugins: [react()], }) ``` ### index.html ```html Vite + React
``` ### src/App.css ```css #root { max-width: 1280px; margin: 0 auto; padding: 2rem; text-align: center; } body { background: linear-gradient(to bottom, #b8aede 0%, #89cff0 100%); } .app-logo { position: absolute; width: 206px; height: 70.93px; top: 0px; left: 60px; padding: 1.47px; gap: 6.88px; color: black; } .logoimage { width: 40px; height: 40px; top: 45px; left: 10px; position: absolute; } .logo { height: 6em; padding: 1.5em; will-change: filter; transition: filter 300ms; } .logo:hover { filter: drop-shadow(0 0 2em #646cffaa); } .logo.react:hover { filter: drop-shadow(0 0 2em #61dafbaa); } @keyframes logo-spin { from { transform: rotate(0deg); } to { transform: rotate(360deg); } } @media (prefers-reduced-motion: no-preference) { a:nth-of-type(2) .logo { animation: logo-spin infinite 20s linear; } } .card { padding: 2em; } .read-the-docs { color: #888; } ``` ### src/index.css ```css :root { font-family: Inter, system-ui, Avenir, Helvetica, Arial, sans-serif; line-height: 1.5; font-weight: 400; color-scheme: light dark; color: rgba(255, 255, 255, 0.87); background-color: #242424; font-synthesis: none; text-rendering: optimizeLegibility; -webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale; } a { font-weight: 500; color: #646cff; text-decoration: inherit; } a:hover { color: #535bf2; } body { margin: 0; display: flex; place-items: center; min-width: 320px; min-height: 100vh; } h1 { font-size: 3.2em; line-height: 1.1; } button { border-radius: 8px; border: 1px solid transparent; padding: 0.6em 1.2em; font-size: 1em; font-weight: 500; font-family: inherit; background-color: #1a1a1a; cursor: pointer; transition: border-color 0.25s; } button:hover { border-color: #646cff; } button:focus, button:focus-visible { outline: 4px auto -webkit-focus-ring-color; } @media (prefers-color-scheme: light) { :root { color: #213547; background-color: #ffffff; } a:hover { color: #747bff; } button { background-color: #f9f9f9; } } ``` ### src/components/PptxReader.css ```css #root { max-width: 1280px; margin: 0 auto; padding: 2rem; text-align: center; } body { color: black } .topic { text-align: center; width: 1129px; height: 600px; top: 110px; border-radius: 25px; background-color: white; padding: 10px; box-shadow: 0px 8px 16px 0px rgba(0, 0, 0, 0.5); border-radius: 25px; padding: 10px; display: flex; align-items: center; justify-content: center; flex-direction: column; } .upload { left: 0px; } .once { left: 0px; } .fileUpload { background-color: #f7f7f7; width: 800px; height: 300px; display: block; align-items: center; justify-content: center; display: flex; align-items: center; justify-content: center; flex-direction: column; border-radius: 15px; } .cloud { width: 68.95px; height: 68.95px; display: inline-block; vertical-align: middle; /* This will vertically center the icon */ position: relative; /* Relative positioning to move it slightly upwards */ top: -10px; /* Adjust this value to move the icon up by the desired amount */ } .topic { background-color: white; } .container { display: flex; justify-content: space-around; /* Adjusts the spacing around items */ } .a1::before, .a2::before, .a3::before, .a4::before { content: url('src/components/Vector.png'); /* Replace 'path/to/your/image.png' with the actual path to your image */ margin-right: 10px; /* Adjusts the space between the image and the text */ margin-left: 10px; vertical-align: middle; /* Adjusts the alignment of the image relative to the text */ } ``` ### src/components/PptxReader.jsx ```javascript import React, { useState } from 'react'; import JSZip from 'jszip'; // Import JSZip library import axios from 'axios'; import './PptxReader.css'; const PptxReader = () => { const [file, setFile] = useState(null); const [text, setText] = useState([]); const [topic, setTopic] = useState(''); function getTextFromNodes(node, tagName, namespaceURI, isExtractingSlideNotes = false) { if(!isExtractingSlideNotes){ let currSlideText = {} let slideBulletPoints = [] const textNodes = node.getElementsByTagNameNS(namespaceURI, tagName); if(textNodes.length == 0) return currSlideText; currSlideText["title"] = textNodes[0].textContent.trim(); for (let i = 1; i < textNodes.length; i++) { slideBulletPoints.push(textNodes[i].textContent.trim()); } currSlideText["bullet_points"] = slideBulletPoints; return currSlideText; } else{ const textNodes = node.getElementsByTagNameNS(namespaceURI, tagName); if(textNodes.length == 0) return []; let currSpeakerNotes = []; for (let i = 0; i < textNodes.length; i++) { currSpeakerNotes.push(textNodes[i].textContent.trim()); } return currSpeakerNotes; } } const handleFileChange = (e) => { const selectedFile = e.target.files[0]; setFile(selectedFile); }; const handleFileUpload = async (e) => { if (!file) return; e.preventDefault(); const reader = new FileReader(); reader.onload = async (event) => { const arrayBuffer = event.target.result; const zip = await JSZip.loadAsync(arrayBuffer); const aNamespace = "http://schemas.openxmlformats.org/drawingml/2006/main"; let allSlidesText = [] let slideIndex = 1; while (true) { const slideFile = zip.file(`ppt/slides/slide${slideIndex}.xml`); const notesFile = zip.file(`ppt/notesSlides/notesSlide${slideIndex}.xml`); if (!slideFile) break; const slideXmlStr = await slideFile.async('text'); const notesXmlStr = await notesFile.async('text'); const parser = new DOMParser(); const slidesXmlDoc = parser.parseFromString(slideXmlStr, 'application/xml'); const notesXmlDoc = parser.parseFromString(notesXmlStr, 'application/xml'); let currSlideDict = getTextFromNodes(slidesXmlDoc, "t", aNamespace); currSlideDict["speaker_notes"] = getTextFromNodes(notesXmlDoc, "t", aNamespace); allSlidesText.push(currSlideDict); slideIndex++; } setText(allSlidesText); setTopic(allSlidesText[0]["title"]) console.log(text); console.log(topic); try { // Send data to Flask server const response = await axios.post('http://localhost:5000/api/endpoint', {'topic' : topic, 'slides_data': allSlidesText}); console.log('Response from server:', response.data); // Handle response as needed } catch (error) { console.error('Error sending data:', error); // Handle error } return allSlidesText; }; reader.readAsArrayBuffer(file); }; return (

Upload a Powerpoint

Once you upload, our AI will turn your slides into a video presentation.

Help students catch up

Create online courses

Create training videos

Done within 1 min

); }; export default PptxReader; ```