Project Info
This project did not submit a demo video on Devpost.
Problem There are two sides to technological progress: a brighter, progressive future where new technology becomes further integrated into our lives and the thousands of old devices that fall into disuse to welcome the new change. As we approach another technological frontier with the advancement of AI, it is often easy to look at the bright future ahead, but it is far more important to see what and more importantly, who we leave behind. Low-quality videos and pictures often fall to the wayside despite the significant amount of childhood memories they hold. We aim to bring back these memories through our software. However, even more importantly, the advancement of technology can leave people behind. Limits on the availability of good graphics can worsen education, restrict communication, and widen disparities for people of lower socioeconomic status. We aim to bridge these disparities by bringing video enhancement to all.
Inspiration
: Our journey to creating this suite of video enhancement tools stemmed from a critical realization: video content is not one-size-fits-all. By recognizing the diverse needs of audiences including live streamers, individuals who are colorblind, the elderly, and film preservationists, in addition to the general population, we set out to dismantle the barriers to digital content accessibility. We aim to harness the latest advancements in AI to elevate video quality and ensure that everyone, regardless of their age, socioeconomic status, or disability, can fully engage with and appreciate the richness of digital media.
What it does
: Sapheneia utilizes open-source video enhancement technology, we have developed a straightforward web interface that significantly simplifies the process of improving video quality. This platform represents a leap forward in our mission to make high-quality video content accessible to everyone. Centralized Video Enhancement Hub: Our platform provides a seamless experience for users looking to enhance their video content. By uploading a video directly to our website, users can witness the transformation of their graphics in real-time. The interface thoughtfully displays the enhanced video alongside the original, enabling immediate comparison and showcasing the dramatic improvements our technology delivers. How it works: The Bread and Butter: Our core technology revolves around an AI-driven approach specifically designed for enhancing animated visuals. This model is adept at analyzing low-resolution footage and predicting a high-quality output by filling in missing details, reducing noise, and sharpening lines without compromising the animated original style. Trained on a comprehensive dataset of animated content, it ensures that each frame is not just clearer but also in line with the intent of the original creation. At Sapheneia, our infrastructure is built around a powerful pipeline hosted on AWS Cloud, which serves as the foundation for our vision enhancement model. This setup enables our web interface to interact seamlessly with our backend. Here, videos are spliced and sent to the server, where they undergo real-time enhancement. Once processed, the enhanced video is promptly returned to the interface, ready to be displayed. What’s Next: Dim Scene Enhancement: A challenge our currently implemented model faces is that with darker scenes, it ranks poorly in video enhancement because the model finds it difficult to find the contrast lines of buildings, faces, etc, and can’t accurately differentiate and enhance different artifacts in the frame. This algorithm will isolate these darker scenes, temporarily increase their brightness for more effective enhancement, and then restore the original brightness level before presenting the final, enhanced video to the user. This method aims to improve the model's ability to enhance details in low-light conditions, ensuring better video quality regardless of scene brightness. Content-Type Enhancement: Understanding the distinct characteristics of animated versus live-action footage, we're introducing a user-friendly feature that allows for the selection between enhancement modes tailored to each type. This approach enhances the viewing experience by applying model adjustments that are most appropriate for the content's nature. Our Market: Our suite of tools is aimed at a broad user base, from individuals looking for a better viewing experience to live streamers and film restorationists seeking to enhance their video quality. Due to the accessibility and affordability of our product, we bring an attractive alternative to a new market of people that has been blocked off by paywalls and hardware limitations.We envision our largest market opportunity in licensing collaborations with leading live content creation platforms, scaling our reach and impact post-beta testing with companies such as Twitch, Tiktok, & Youtube Gaming. For an exploration of additional applications, including the transformative use of our real-time vision enhancement technology in various settings, please scroll down to the Proof of Concept section. Differences from our Competitors: The biggest advantage we have over our competitors is accessibility. We unlock the technology for the public - developers and non-developers alike. Our software program is downloadable by anyone with access to a computer and is completely free. In comparison, people choosing to opt-into services like RTX or Topaz Labs are locked behind a paywall that can scale as high as $300 per tool and hardware components such as a graphics card or large amounts of RAM to be used. Making our product more inclusive and available to a broader audience, enabling individuals from various economic backgrounds to access and utilize it. Proof of Concept @ TreeHacks: Reazon Holdings: At Sapheneia, we found a unique opportunity to leverage our video enhancement technology in collaboration with Reazon's venture into anime production. This partnership not only aims to elevate the quality of new anime but also rejuvenates older titles, sparking renewed interest and enabling game development around these refreshed IPs. The growing demand for revitalizing early 2000s anime aligns perfectly with our capabilities, positioning us to significantly impact Reazon's diverse portfolio by enhancing viewer engagement and expanding their market reach. Verdaka: Known for their innovative use of generative AI to anonymize individuals in video footage by blurring faces, is set to enhance its capabilities further through our collaboration. Building on the momentum of our successful hackathon project, we are developing a real-time plugin designed to significantly upscale video quality for both security footage and playback scenarios. This initiative aims to refine visual clarity without compromising privacy, ensuring that security surveillance and video playback are both sharp and secure. Our willingness to collaborate represents a leap forward in combining privacy with high-definition video technology, setting a new standard for security and playback solutions. Parrot Drones: A leader in creating drones operable from any global location with cellular data, offers an exciting opportunity through its open SDK. By which we aim to leverage this by introducing a real-time video quality enhancement plugin that boosts the clarity of footage captured by these drones. This innovation is not just a leap in video technology; it has significant social implications. Enhanced video quality becomes a crucial asset in missions like searching for survivors in disaster-stricken or conflict-affected zones. Moreover, our approach ingeniously navigates the challenges of low-bandwidth areas by allowing the capture of lower-quality video, which we then upscale on the receiver's end where better internet connectivity is present. This ensures that critical visual data remains accessible and clear, maximizing the drones' utility in vital humanitarian and surveillance efforts. Business Plan: Our approach centers on offering a range of free, user-friendly tools that democratize access to cutting-edge video enhancement technology, ensuring minimal to no barriers for the average user. Through this approach, we can reach out to a wider and more interested audience that is looking for video-enhancement software because of the equitable and accessible nature of our software. This means we can bridge the gaps brought about by socioeconomic differences and bring video enhancement to the ones who need and want it most at a much more affordable price. We plan to access this market through an extensive advertising campaign and a two-month free trial period where all of our premium features will be available to the wider market. This would include the live-streaming feature as well as more specific models that are tailored for the consumption of certain forms of media like anime. Our general video enhancement software, however, would be available to the wider public. Our objective is to attract a significant volume of online traffic to our extensions and plugins, leveraging this momentum as detailed in our "Next Steps" section. This pivotal phase will enable us to initiate outreach and forge connections with the companies highlighted in our Proof of Concept and with leading live content platforms. Our goal is to cultivate partnerships that facilitate the seamless integration of our video enhancement technology, thereby elevating the quality and accessibility of video content across the digital landscape.
Sharp-AI-ly
Engineered a dockerized environment of an open-source REAL ESRGAN model ↗ interfaced with a FlaskAPI on an AWS EC2 p2.xlarge instance, ensuring cost-effective scalability for GPU-intensive tasks through NVIDIA’s TensorRT SDK, leading to video enhancement in 2x time. Submitted to TreeHacks 2024 ↗
Demo
https://github.com/nairvishnumail/Sharp-ai-ly/assets/27198773/c8b4ea97-988f-40ba-a6bb-5c9e5e5a19df
AWS Deploy Steps
In order to deploy on AWS, you need to make sure that the AWS account has at least 4 vCPUs available. If you do not have at least 4 vCPUs, you can request more here. Provide a reasonable explanation as to why you need it and within 24-48 hours you should have access to it.
EC2 launch steps:
-
Launch an EC2 instance
-
Choose an appropriate AMI (important) - I chose Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.0.1 (Amazon Linux 2) 20240206
-
Choose p2.xlarge instance type. G instances do not work with the above AMI. But, feel free to try different types.
-
Create a security group to allow SSH traffic ("anywhere" is good for testing, but not recommended due to low security)
-
Choose 45 GB gp3 storage
WARN: AWS EC2 instances that need NVIDIA GPUs can be quite costly, so make sure you terminate the instance once you're done working with it.
SSH & Setup NVIDIA drivers
- ssh into the instance by clicking on "connect" and following the ssh steps
- Follow steps here to setup NVIDIA SMI on your EC2 instance. This step is hard to get right since there are many different guides.
Setting up server
Follow the self-deploy steps below on the instance to have the server up and working.
Self-deploy steps
NOTE: To self-deploy, your machine needs to have an NVIDIA GPU running with the correct drivers. Make sure
nvidia-smiworks correctly.
git clone https://github.com/nairvishnumail/Sharp-ai-ly.git- Make sure you have docker and docker-compose: Get it here
- Dockerize input and output directories
docker run -v "<project_path>/Sharpr-ai-ly/src/backend/ai/input:/input" -v "<project_path>/Sharpr-ai-ly/src/backend/ai/out:/out" -it docker-compose run --rm vsgan_tensorrtpython app.py
You now have the server running on at localhost:8080
Send a POST request with video_url in the body to http://localhost:8080/process-video to get back a S3 object URL for the processed video
For tunnelling a localhost environment, I used ngrok that helped me hit the endpoint from different PCs
Analysis
View
Metric
- 4
- 3
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- HTMLIn code
- JavaScriptIn code
- PythonIn code
- ReactIn code
- Tailwind CSSIn code
- C++Claimed
- DockerClaimed
- Node.jsClaimed
6 of 9 appear in the indexed code. 3 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
18 KB
Source files
17
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
vvnu0/SharprAI-TreeHacks
32 files · 2.5 MB · @ eba3d3f
Structure
Application logic
13 files · 41%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- Markdown34%
- Python23%
- JavaScript23%
- HTML9%
- YAML6%
- CSS5%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
frontend/package.json
npm · 8- @testing-library/jest-dom
- @testing-library/react
- @testing-library/user-event
- react
- react-dom
- react-scripts
- web-vitals
- +1 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.