# Project export: Flipside This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: TreeHacks 2024 - Tagline: News reimagined: Swipe to see every side of the story. - Devpost: https://devpost.com/software/flipside-rz9i1v - GitHub: https://github.com/robbym-dev/Flipside-AI - Video: https://www.youtube.com/embed/z3GYn6hIg6A?enablejsapi=1&hl=en_US&rel=0&start=&version=3&wmode=transparent - Result: winner (InterSystems: Best Use of GenAI using InterSystems IRIS Vector Search ($2k Cash [1st] & $1.5k Cash [2nd] & $1k Cash [3rd])) - Team: 3 GitHub contributor(s) — ayaanmalik04 (5 commits), jaisal-wispr (3 commits), Robby Manihani (2 commits) ## Devpost submission (written by the team) ### Inspiration The way we consume news is often one-dimensional, confined within our own echo chambers without even knowing how to find opposing viewpoints. Don't you want to know what the other side thinks? As bot-infested and hate-speech-filled social media platforms become primary news sources, students fall victim to misinformation and hidden agendas. We must prevent the spread of extremism to our generation by providing students with a safe space to stay informed. Our mission is not just to redefine the news experience, but to bring people together by fostering understanding and discourse between polarized groups. What Flipside does Flipside breaks through echo chambers and creates a safe space for students to discover the world through multiple lenses. Glide through bite-sized news stories in a sleek interface. Start with an AI-generated, unbiased article, then swipe right to explore contrasting viewpoints sourced from real people on online forums, Twitter posts and YouTube comment sections. If the story captures your imagination, enter the 'Rabbithole' and deepen your understanding through a guided conversation with our purpose-built AI agent that provides personalized and safe answers. Key Features 1. Bite-sized News: Swipe up and down to scroll through story summaries, swipe left/right to dig deeper into a story that catches your eye. Kinda like Tinder 👀 2. Explore Narratives: Read thoughts, reactions and opinions on any story sourced from Reddit, Twitter and YouTube. A place for every perspective. 3. Enter The Rabbithole: Ever been down a Wikipedia rabbit hole? Yeah, it's pretty addicting. Our rabbit hole features an AI-generated wiki fine-tuned on the topic with a trove of real-time information on the news story. You can also chat with a personalized AI agent to learn more. 4. Subscribe to Stories, not Sources: Keep yourself out of the echo chamber by following news stories instead of sources. As soon as there's an update to your favorite news story, we'll send you a notification. ### How we built it 1. Automatic Data Curation: Our Mailman Agent This agent is the crux of our app. It’s tasked to periodically fetch latest news to keep our app on top of the headlines while also notifying users if they might be interested in the update. Our Fetch AI agent uses time and geolocation to fetch the top headlines from the NewsAPI and process it through our extensive Together.ai GenAI content generation pipeline and publish it on our app. During this process, it uses llama_index from InterSystems IRIS to generate keywords through a RAG model, fed into Twitter Search API and webscrapers to extract relevant discussions. All data is automatically stored on and retrieved from Firebase Firestore. 2. Flipside Mobile App We build our platform using React Native and Swift for iOS. We used complex gesture handling to enable an intuitive and addictive swipe & scroll based user experience. We set up a Flask server to communicate with the various systems deployed. On top of this, we built a beautiful UI to ensure user experience was exactly how we imagined it. 3. Rabbithole Mode This agent provides the seamless replies in the Rabbithole mode. The agent is tasked to complete a whole AI pipeline to curate the best reply semantically to the context and accurately to the query. We utilized Fetch.ai’s versatile AI agent framework in Python to retrieve requests from our React Native app containing our query and a Together AI fine-tuned model. This is further processed using llama_index integration with the IRIS container by InterSystems for a RAG model powered by our context file and query intertwined with an engineered prompt. We utilize certain criteria such as query relevance, frequency, distinctiveness and contextual relevance. These keywords are fed into NewsAPI to retrieve the most similar articles which are added as context to generate valid reliable information. 4. Vector Search We utilize a semantic SQL vector search powered by InterSystems at the heart of our prediction tasks throughout the app. There were two main tasks with this foundation: 1. Subscriptions: We maintain a database of subscribed articles of each user. We take in a new article fetched by our Mailman Agent powered by Fetch.ai and generate embeddings of the fetched article and all subscribed articles using HuggingFace’s all-mpnet-base-v2. We then calculate a normalized vector dot product, sorted in decreasing order, and choose the top 5 users above a 0.75 threshold. This informs us that these users are most-likely to be interested in the new updates and we send them a notification. 2. Flipside Companion: We utilize the same process in our Chrome Extension to give a bit-sized summary of the article while providing the different narratives of the current article by choosing the top two articles from our database, ordered by a vector dot product (SQL semantic search) by InterSystems of the current article and our database. 5. Prompt Engineering: The Magic ### Challenges we ran into Jaisal: AI agents, vector databases, and the world's most powerful LLMs; one would assume that swiping would be the easiest feature to implement - wrong. Why React why? Why did this take 8 hours to build? Robby: I spent quite a while trying to get an LLM to respond to my prompts accurately. In my exasperation, I asked it to take a deep breath and then it started working! Ayaan: Found a weird bug while implementing the InterSystems IRIS SQL Search and spent 4 hours with the InterSystems team to figure out this never-seen-before bug! Anant: Going from a sleep-deprived midterm week to a sleep-deprived hackathon weekend :) ### Accomplishments we're proud of Ayaan: I'm so proud of Robby for taking ownership of some of the most tedious and technically difficult tasks we had, plus for being a fantastic group DJ. Robby: I'm so proud of Ayaan for debugging one of our most complex modules, and as we hit enter in the terminal to test it...his laptop died! Jaisal: I'm so proud of Anant for coming up with great ideas for Flipside's features, combined with some really catchy names to make our project memorable. Anant: I'm so proud of Jaisal for managing to build out the entire UI, creating the technical outline for our app and doing so much more while helping the rest of us through our first hackathon. Truly incredible. ### What we learned 36 hours is a LOT of time wow... We actually built one of our most complex projects yet - even calling it full stack won't cut it. We learnt the important balance of compromise: with dozens of ideas we had to prioritize the ones which were important (healthy arguments!), settle for imperfect implementations, and think about the tech, the business, the moral implications, and most importantly, the user experience. These are all skills that we don't get the practice in our daily lives. So, Treehacks was the perfect place to get exposed to what we would say is a more realistic representation of the real world. And we developed newfound appreciation for each other. The heart-to-heart conversations you have on 2 hours of sleep go a long way. ### What's next News on the go: We already have AI generated images (did you notice? :P). Next it's time for AI podcasts, weekly summaries and rabbithole-y stories - all automatically and perfectly generated. Highlight and Reply: A lot of users are refering to a small piece of text within the article when the make comments. Why not add the option to quote pieces of text? See which parts are interesting to others. Kind of like the YouTube "Most Watched Part" feature. ## README (from the GitHub repository) ## Flipside: Best GenAI Hack (First Prize) @ TreeHacks 2024 ## Inspiration The way we consume news is often one-dimensional, confined within our own echo chambers without even knowing how to find opposing viewpoints. Don't you want to know what the other side thinks? As bot-infested and hate-speech-filled social media platforms become primary news sources, students fall victim to misinformation and hidden agendas. We must prevent the spread of extremism to our generation by providing students with a safe space to stay informed. Our mission is not just to redefine the news experience, but to bring people together by fostering understanding and discourse between polarized groups. ## What Flipside does Flipside breaks through echo chambers and creates a safe space for students to discover the world through multiple lenses. Glide through bite-sized news stories in a sleek interface. Start with an AI-generated, unbiased article, then swipe right to explore contrasting viewpoints sourced from real people on online forums, Twitter posts and YouTube comment sections. If the story captures your imagination, enter the 'Rabbithole' and deepen your understanding through a guided conversation with our purpose-built AI agent that provides personalized and safe answers. ## Key Features **1. Bite-sized News:** Swipe up and down to scroll through story summaries, swipe left/right to dig deeper into a story that catches your eye. Kinda like Tinder 👀 **2. Explore Narratives:** Read thoughts, reactions and opinions on any story sourced from Reddit, Twitter and YouTube. A place for every perspective. **3. Enter The Rabbithole:** Ever been down a Wikipedia rabbit hole? Yeah, it's pretty addicting. Our rabbit hole features an AI-generated wiki fine-tuned on the topic with a trove of real-time information on the news story. You can also chat with a personalized AI agent to learn more. **4. Subscribe to Stories, not Sources:** Keep yourself out of the echo chamber by following news stories instead of sources. As soon as there's an update to your favorite news story, we'll send you a notification. ## How we built it **1. Automatic Data Curation: Our Mailman Agent** This agent is the crux of our app. It’s tasked to periodically fetch latest news to keep our app on top of the headlines while also notifying users if they might be interested in the update. Our Fetch AI agent uses time and geolocation to fetch the top headlines from the NewsAPI and process it through our extensive Together.ai GenAI content generation pipeline and publish it on our app. During this process, it uses llama_index from InterSystems IRIS to generate keywords through a RAG model, fed into Twitter Search API and webscrapers to extract relevant discussions. All data is automatically stored on and retrieved from Firebase Firestore. **2. Flipside Mobile App** We build our platform using React Native and Swift for iOS. We used complex gesture handling to enable an intuitive and addictive swipe & scroll based user experience. We set up a Flask server to communicate with the various systems deployed. On top of this, we built a beautiful UI to ensure user experience was exactly how we imagined it. **3. Rabbithole Mode** This agent provides the seamless replies in the Rabbithole mode. The agent is tasked to complete a whole AI pipeline to curate the best reply semantically to the context and accurately to the query. We utilized Fetch.ai’s versatile AI agent framework in Python to retrieve requests from our React Native app containing our query and a Together AI fine-tuned model. This is further processed using llama_index integration with the IRIS container by InterSystems for a RAG model powered by our context file and query intertwined with an engineered prompt. We utilize certain criteria such as query relevance, frequency, distinctiveness and contextual relevance. These keywords are fed into NewsAPI to retrieve the most similar articles which are added as context to generate valid reliable information. **4. Vector Search** We utilize a semantic SQL vector search powered by InterSystems at the heart of our prediction tasks throughout the app. There were two main tasks with this foundation: **1. Subscriptions**: We maintain a database of subscribed articles of each user. We take in a new article fetched by our Mailman Agent powered by Fetch.ai and generate embeddings of the fetched article and all subscribed articles using HuggingFace’s all-mpnet-base-v2. We then calculate a normalized vector dot product, sorted in decreasing order, and choose the top 5 users above a 0.75 threshold. This informs us that these users are most-likely to be interested in the new updates and we send them a notification. **2. Flipside Companion**: We utilize the same process in our Chrome Extension to give a bit-sized summary of the article while providing the different narratives of the current article by choosing the top two articles from our database, ordered by a vector dot product (SQL semantic search) by InterSystems of the current article and our database. **5. Prompt Engineering: The Magic** ## Detected evidence (automated analysis) Indexed codebase: 42 recognized source files, 98 KB. - C (language) — detected in the code - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Swift (language) — detected in the code - TypeScript (language) — detected in the code - Firebase (technology) — claimed on Devpost, not found in the code - Flask (technology) — claimed on Devpost, not found in the code - Hugging Face (technology) — claimed on Devpost, not found in the code - LlamaIndex (technology) — claimed on Devpost, not found in the code - OpenAI (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (66 of 66) ``` .DS_Store Backend Pipeline/flask_app.py Backend Pipeline/intersys.py Backend Pipeline/keywords.py Backend Pipeline/mailman_agent.py Backend Pipeline/notify_users.py Backend Pipeline/rabbit_agent.py Chrome Extension/content.js Chrome Extension/manifest.json Chrome Extension/popup.html Chrome Extension/popup.js Chrome Extension/styles.css Flipside Native/app.json Flipside Native/App.tsx Flipside Native/babel.config.js Flipside Native/components/AudioPlayer/index.tsx Flipside Native/components/BottomBar/index.js Flipside Native/components/BottomSheet/index.tsx Flipside Native/components/FullItem/FullItem.tsx Flipside Native/components/ImageSlider/index.tsx Flipside Native/components/Narrative/Narrative.tsx Flipside Native/components/Narratives/Narratives.tsx Flipside Native/components/Rabbithole/Rabbithole.tsx Flipside Native/components/RoundedImage/index.tsx Flipside Native/components/SummaryItem/SummaryItem.tsx Flipside Native/components/TextScroll/index.tsx Flipside Native/components/TextView/index.tsx Flipside Native/components/TextViewNavigator/index.tsx Flipside Native/components/TopBar/index.tsx Flipside Native/components/VideoList/styles.js Flipside Native/components/VideoList/VideoList.tsx Flipside Native/Gemfile Flipside Native/Gemfile.lock Flipside Native/index.js Flipside Native/ios/Flipside.xcodeproj/project.pbxproj Flipside Native/ios/Flipside.xcodeproj/project.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist Flipside Native/ios/Flipside.xcodeproj/project.xcworkspace/xcuserdata/jaisalwispr.xcuserdatad/UserInterfaceState.xcuserstate Flipside Native/ios/Flipside.xcodeproj/xcshareddata/xcschemes/Clipverse.xcscheme Flipside Native/ios/Flipside.xcworkspace/contents.xcworkspacedata Flipside Native/ios/Flipside.xcworkspace/xcshareddata/IDEWorkspaceChecks.plist Flipside Native/ios/Flipside.xcworkspace/xcuserdata/jaisalwispr.xcuserdatad/UserInterfaceState.xcuserstate Flipside Native/ios/Flipside/AppDelegate.h Flipside Native/ios/Flipside/AppDelegate.mm Flipside Native/ios/Flipside/Images.xcassets/AppIcon.appiconset/Contents.json Flipside Native/ios/Flipside/Images.xcassets/Contents.json Flipside Native/ios/Flipside/Info.plist Flipside Native/ios/Flipside/LaunchScreen.storyboard Flipside Native/ios/Flipside/main.m Flipside Native/ios/FlipsideTests/ClipverseTests.m Flipside Native/ios/FlipsideTests/Info.plist Flipside Native/ios/HandleAudio.swift Flipside Native/ios/Podfile Flipside Native/ios/Podfile.lock Flipside Native/jest.config.js Flipside Native/metro.config.js Flipside Native/package.json Flipside Native/pages/Course/index.js Flipside Native/pages/FeedContext.js Flipside Native/pages/Home/index.js Flipside Native/pages/Profile/index.js Flipside Native/pages/Saved/index.js Flipside Native/README.md Flipside Native/service.js Flipside Native/tsconfig.json llm_processing.py README.md ``` ### Dependencies - Flipside Native/package.json: @babel/core@^7.20.0, @babel/preset-env@^7.20.0, @babel/runtime@^7.20.0, @flyerhq/react-native-chat-ui@^1.4.3, @gorhom/bottom-sheet@^4, @react-native-assets/slider@^7.0.3, @react-native-community/slider@^4.4.3, @react-native-firebase/app@^18.8.0, @react-native-firebase/firestore@^18.8.0, @react-native/eslint-config@^0.72.2, @react-native/metro-config@^0.72.11, @react-navigation/bottom-tabs@^6.5.8, @react-navigation/material-top-tabs@^6.6.6, @react-navigation/native@^6.1.10, @react-navigation/stack@^6.3.17, @tsconfig/react-native@^3.0.0, @types/react@^18.0.24, @types/react-native-video@^5.0.15, @types/react-test-renderer@^18.0.0, axios@^1.6.7, babel-jest@^29.2.1, eslint@^8.19.0, jest@^29.2.1, metro-react-native-babel-preset@0.76.8, prettier@^2.4.1, react@18.2.0, react-native@0.72.4, react-native-collapsible@^1.6.1, react-native-feather@^1.1.2, react-native-fs@^2.20.0, react-native-gesture-handler@^2.12.1, react-native-image-slider-box@^2.0.7, react-native-image-slider-carousel@^1.1.0, react-native-pager-view@^6.2.3, react-native-progress@^5.0.0, react-native-reanimated@^3.4.2, react-native-reanimated-carousel@^3.5.1, react-native-safe-area-context@^4.9.0, react-native-screens@^3.29.0, react-native-sound@^0.11.2, react-native-svg@^13.13.0, react-native-tab-view@^3.5.2, react-native-track-player@^3.2.0, react-native-video@^5.2.1, react-test-renderer@18.2.0, typescript@4.8.4 ### Recent commits (newest first) - Update README.md - Create README.md - Delete Flipside Native/ios/GoogleService-Info.plist - Add files via upload - Add React App - Add folder - Delete Chrome Extension/pic.png - Add files via upload - Delete Backend Pipeline/chroma.py - Add files via upload - Delete Backend Pipeline directory - Add files via upload ## Key source files (fetched from GitHub, selected and truncated for size) ### Flipside Native/Gemfile ``` source 'https://rubygems.org' # You may use http://rbenv.org/ or https://rvm.io/ to install and use this version ruby ">= 2.6.10" gem 'cocoapods', '~> 1.12' ``` ### Flipside Native/package.json ``` { "name": "Clipverse", "version": "0.0.1", "private": true, "scripts": { "android": "react-native run-android", "ios": "react-native run-ios", "lint": "eslint .", "start": "react-native start", "test": "jest" }, "dependencies": { "@flyerhq/react-native-chat-ui": "^1.4.3", "@gorhom/bottom-sheet": "^4", "@react-native-assets/slider": "^7.0.3", "@react-native-community/slider": "^4.4.3", "@react-native-firebase/app": "^18.8.0", "@react-native-firebase/firestore": "^18.8.0", "@react-navigation/bottom-tabs": "^6.5.8", "@react-navigation/material-top-tabs": "^6.6.6", "@react-navigation/native": "^6.1.10", "@react-navigation/stack": "^6.3.17", "axios": "^1.6.7", "react": "18.2.0", "react-native": "0.72.4", "react-native-collapsible": "^1.6.1", "react-native-feather": "^1.1.2", "react-native-fs": "^2.20.0", "react-native-gesture-handler": "^2.12.1", "react-native-image-slider-box": "^2.0.7", "react-native-image-slider-carousel": "^1.1.0", "react-native-pager-view": "^6.2.3", "react-native-progress": "^5.0.0", "react-native-reanimated": "^3.4.2", "react-native-reanimated-carousel": "^3.5.1", "react-native-safe-area-context": "^4.9.0", "react-native-screens": "^3.29.0", "react-native-sound": "^0.11.2", "react-native-svg": "^13.13.0", "react-native-tab-view": "^3.5.2", "react-native-track-player": "^3.2.0", "react-native-video": "^5.2.1" }, "devDependencies": { "@babel/core": "^7.20.0", "@babel/preset-env": "^7.20.0", "@babel/runtime": "^7.20.0", "@react-native/eslint-config": "^0.72.2", "@react-native/metro-config": "^0.72.11", "@tsconfig/react-native": "^3.0.0", "@types/react": "^18.0.24", "@types/react-native-video": "^5.0.15", "@types/react-test-renderer": "^18.0.0", "babel-jest": "^29.2.1", "eslint": "^8.19.0", "jest": "^29.2.1", "metro-react-native-babel-preset": "0.76.8", "prettier": "^2.4.1", "react-test-renderer": "18.2.0", "typescript": "4.8.4" }, "engines": { "node": ">=16" } } ``` ### Flipside Native/index.js ```javascript /** * @format */ import {AppRegistry} from 'react-native'; import App from './App'; import {name as appName} from './app.json'; import TrackPlayer from 'react-native-track-player'; AppRegistry.registerComponent(appName, () => App); TrackPlayer.registerPlaybackService(() => require('./service')); ``` ### Flipside Native/App.tsx ```typescript import React from 'react'; import { NavigationContainer } from '@react-navigation/native'; import { createBottomTabNavigator } from '@react-navigation/bottom-tabs'; import { GestureHandlerRootView } from 'react-native-gesture-handler'; import CustomTabBar from './components/BottomBar'; import HomeScreen from './pages/Home'; import CourseScreen from './pages/Course'; import SavedScreen from './pages/Saved'; import ProfileScreen from './pages/Profile'; const Tab = createBottomTabNavigator(); function App(): JSX.Element { return ( } screenOptions={{ headerShown: false }}> ); } export default App; ``` ### Flipside Native/pages/Saved/index.js ```javascript import React from 'react'; import { View, Text } from 'react-native'; const SavedScreen = () => { return ( Saved ); }; export default SavedScreen; ``` ### Flipside Native/pages/Profile/index.js ```javascript import React from 'react'; import { View, Text } from 'react-native'; const ProfileScreen = () => { return ( Profile ); }; export default ProfileScreen; ``` ### Flipside Native/pages/Home/index.js ```javascript import React from 'react'; import { View, Text, StatusBar, useColorScheme,} from 'react-native'; import BottomSheet from '../../components/BottomSheet'; import { Colors } from 'react-native/Libraries/NewAppScreen'; import TextScroll from '../../components/TextScroll'; const HomeScreen = () => { const isDarkMode = useColorScheme() === 'dark'; const backgroundStyle = { backgroundColor: isDarkMode ? Colors.darker : Colors.lighter, }; return ( Home ); }; export default HomeScreen; ``` ### Flipside Native/components/ImageSlider/index.tsx ```typescript import { View, Dimensions, FlatList, Image } from "react-native"; import RoundedImage from "../RoundedImage"; const { width } = Dimensions.get("window"); const ImageSlider = (props: {posts: any[], id: string}) => { const imageData = props.posts.map((post) => { return { timestamp: post.ms, text: post.phrase, }; }); return ( "key" + index} horizontal pagingEnabled renderItem={({ item }) => { return ( ); }} style={{ width: width, height: 380, marginVertical: 10 }} snapToAlignment={"start"} decelerationRate={"fast"} snapToInterval={300 + 10} disableIntervalMomentum={true} /> ); }; export default ImageSlider; ``` ### Flipside Native/components/RoundedImage/index.tsx ```typescript import React, {useEffect, useState} from 'react'; import { View, Image, Text } from 'react-native'; import storage from '@react-native-firebase/storage'; const RoundedImage = (props: {source: string, text: string, id: string}) => { const [imageURL, setimageURL] = useState(''); useEffect(() => { async function getImageURL() { const ref = storage().ref(`creator-content/askvinh/${props.id}/${props.source}.jpg`); const url = await ref.getDownloadURL(); setimageURL(url); } getImageURL(); }, [props.id]); return ( {props.text} ) } export default RoundedImage; ``` ### Flipside Native/pages/Course/index.js ```javascript import React, {useState} from 'react'; import { NavigationContainer } from '@react-navigation/native'; import { createMaterialTopTabNavigator } from '@react-navigation/material-top-tabs'; import { SafeAreaProvider } from 'react-native-safe-area-context'; import BottomSheet from '../../components/BottomSheet'; import Rabbithole from '../../components/Rabbithole/Rabbithole'; import Narratives from '../../components/Narratives/Narratives'; import { useFeed, FeedProvider } from '../FeedContext'; const Tab = createMaterialTopTabNavigator(); function MyTabs() { const { feedId, setFeedId } = useFeed(); return ( ); } export default function CourseScreen() { return ( ); } ``` [31 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]