# Project export: DeepTrends This document was generated by HackStack to give an AI agent context about a hackathon project. Sections are labeled with their provenance; content marked as truncated was cut to keep this document small. ## Project metadata - Hackathon: UC Berkeley AI Hackathon 2025 - Tagline: Track and summarize emerging ML research trends from arXiv using NLP and topic modeling. Auto-generate insightful blog posts with LLMs and chat with our AI assistant—no paper digging needed. - Devpost: https://devpost.com/software/pulseai-8e17zh - GitHub: https://github.com/BrianStormDev/DeepTrends - Team: 3 GitHub contributor(s) — Brian Leong (15 commits), Lucas Stevenson (12 commits), Joey-Tai Phung (5 commits) ## Devpost submission (written by the team) ### Inspiration The rapid pace of machine learning research presents a unique challenge for developers, researchers, and practitioners trying to stay current with the field. With thousands of papers published monthly on arXiv alone, it's nearly impossible to manually track emerging trends, identify breakthrough research, or understand the evolving landscape of ML topics. This information overload inspired us to create DeepTrends—a tool that democratizes access to research insights and makes academic knowledge more accessible to everyone. We recognized a genuine need in the market for an intelligent system that could not only aggregate research but also analyze trends, extract meaningful insights, and present them in an intuitive, digestible format. Our vision was to bridge the gap between cutting-edge research and practical application, helping developers and researchers make informed decisions about which trends to follow and which papers deserve their attention. ### What it does DeepTrends is an ML-powered research trend tracker that automatically discovers, analyzes, and visualizes emerging patterns in machine learning research. The platform: Aggregates research papers from arXiv across key ML domains including artificial intelligence, computer vision, natural language processing, and neural computing Performs intelligent analysis using sentiment analysis on abstracts, BERTopic for topic clustering, and keyword frequency analysis to identify trending themes Generates automated insights by scoring papers based on citations, author reputation, recency, and relevance to popular topics Creates interactive dashboards that display trend evolution over time, topic clustering, and research momentum Produces AI-generated summaries of the most significant papers and emerging research directions Features an intelligent chatbot powered by Anthropic's Claude that answers questions about research trends, explains paper insights, and provides contextual information about the generated blog posts The system maintains a rolling database of the last month's papers, continuously updating to provide real-time insights into the research landscape. ### How we built it DeepTrends combines multiple technologies in a full-stack architecture: Backend (Python/Flask): Built a robust data pipeline using the arXiv API Implemented BERTopic for semantic clustering of research topics from paper abstracts Integrated Hugging Face transformers for sentiment analysis of research content Created a scoring algorithm that weighs papers by citations, author impact, recency, and topic relevance Used SQLite for efficient data storage and retrieval Developed a Flask API to serve processed data to the frontend Frontend (React): Created an interactive dashboard using React with modern UI components Built dynamic data visualizations and trend charts Implemented responsive design with CSS and modern styling Built real-time data fetching and display capabilities AI Integration: Leveraged Anthropic's Claude API for generating intelligent summaries and insights Built an interactive chatbot interface that answers user questions about research trends and paper details Implemented a sliding window chat system for context-aware content generation Used LangChain for document processing and management Data Processing Pipeline: Automated paper collection from specific arXiv categories Real-time trend analysis using frequency counting and semantic similarity Temporal analysis to track topic evolution over time ### Challenges we ran into Web Development Complexity: As a team with stronger ML backgrounds, we faced significant challenges in full-stack web development. Learning React, managing state, and creating responsive UI components required extensive research and iteration. Styling and Design: Working with CSS and achieving a professional, intuitive interface proved more challenging than expected. We had to balance functionality with aesthetic appeal while ensuring cross-browser compatibility. Import Management: Coordinating dependencies across Python backend libraries (BERTopic, transformers, Flask) and JavaScript frontend packages led to numerous compatibility issues that required careful debugging. API Integration: Properly connecting the Flask backend with the React frontend, handling CORS issues, and managing asynchronous data flow presented ongoing technical hurdles. Data Processing Scale: Processing large volumes of arXiv papers in real-time while maintaining responsive performance required optimization of our algorithms and database queries. ### Accomplishments we're proud of Elegant User Interface: Despite the web development challenges, we created a clean, professional-looking dashboard that effectively communicates complex research trends in an intuitive way. Successful arXiv Integration: We mastered the arXiv API and built a robust system that reliably extracts and processes thousands of research papers. Advanced ML Pipeline: Successfully implemented sophisticated NLP techniques including BERTopic clustering and sentiment analysis to extract meaningful insights from academic text, plus an intelligent chatbot that makes research insights accessible through natural conversation. End-to-End Functionality: Built a complete system that takes raw research papers and transforms them into actionable insights, from data collection through visualization. Problem-Solving Resilience: Overcame numerous technical obstacles through persistent debugging, creative solutions, and effective collaboration. ### What we learned This project provided extensive learning across multiple domains: Full-Stack Development: Gained hands-on experience with modern web development, including React component architecture, state management, API integration, and responsive design principles. Team Collaboration: Learned effective strategies for coordinating work across different technical specialties, managing version control conflicts, and integrating diverse code contributions. Machine Learning Applications: Deepened understanding of practical NLP implementation, from theoretical knowledge to production-ready systems handling real-world data. System Architecture: Developed skills in designing scalable data pipelines, managing database operations, and creating efficient data flow between system components. User Experience Design: Appreciated the complexity of translating technical functionality into user-friendly interfaces that effectively communicate insights. ### What's next Expanded Data Sources: Integrate additional academic repositories including Google Scholar, Semantic Scholar, PubMed, and major conference proceedings to provide comprehensive research coverage. Social Intelligence: Monitor ML discussions across social platforms like Twitter, Reddit's r/MachineLearning, and LinkedIn to incorporate community sentiment and real-world impact into trend analysis. Predictive Analytics: Implement time series forecasting using Prophet or ARIMA models to predict emerging research directions before they become mainstream. Advanced Visualization: Create interactive heatmaps, research calendars, and topic evolution networks to provide deeper insights into how research themes develop and interconnect over time. Enhanced Analysis: Expand beyond abstracts to analyze full paper texts where available, providing more nuanced understanding of research contributions and methodologies. Community Features: Build author impact profiles, citation networks, and collaboration patterns to help researchers identify key contributors and potential partnerships in their fields. Personalization: Develop user profiles that learn individual research interests and provide customized trend recommendations and paper suggestions. ## README (from the GitHub repository) # CourseCompass Cal Hacks 2025 Hackathon Project ## Setup 1. Copy the config template: `cp config.template.json config.json` 2. Edit `config.json` and replace `YOUR_API_KEY_HERE` with your actual Claude API key 3. Run the application: `python main.py` # Getting Started with Create React App This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app). ## Available Scripts In the project directory, you can run: ### `npm start` Runs the app in the development mode.\ Open [http://localhost:3000](http://localhost:3000) to view it in your browser. The page will reload when you make changes.\ You may also see any lint errors in the console. ### `npm test` Launches the test runner in the interactive watch mode.\ See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information. ### `npm run build` Builds the app for production to the `build` folder.\ It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes.\ Your app is ready to be deployed! See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information. ### `npm run eject` **Note: this is a one-way operation. Once you `eject`, you can't go back!** If you aren't satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project. Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you're on your own. You don't have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't customize it when you are ready for it. ## Learn More You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started). To learn React, check out the [React documentation](https://reactjs.org/). ### Code Splitting This section has moved here: [https://facebook.github.io/create-react-app/docs/code-splitting](https://facebook.github.io/create-react-app/docs/code-splitting) ### Analyzing the Bundle Size This section has moved here: [https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size](https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size) ### Making a Progressive Web App This section has moved here: [https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app](https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app) ### Advanced Configuration This section has moved here: [https://facebook.github.io/create-react-app/docs/advanced-configuration](https://facebook.github.io/create-react-app/docs/advanced-configuration) ### Deployment This section has moved here: [https://facebook.github.io/create-react-app/docs/deployment](https://facebook.github.io/create-react-app/docs/deployment) ### `npm run build` fails to minify This section has moved here: [https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify](https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify) ## Detected evidence (automated analysis) Indexed codebase: 20 recognized source files, 69 KB. - CSS (language) — detected in the code - HTML (language) — detected in the code - JavaScript (language) — detected in the code - Python (language) — detected in the code - React (technology) — detected in the code - Flask (technology) — claimed on Devpost, not found in the code - Node.js (technology) — claimed on Devpost, not found in the code ## Codebase structure (from repository index) ### Files (26 of 26) ``` .gitignore backend/app.py backend/database.py backend/db.db backend/fast_cache/arxiv_data_90_2025-06-22.pkl backend/helper_classes.py backend/ManualSlidingWindowChat.py backend/test.py backend/trend_viz.py my-app/author.py my-app/main.py my-app/package.json my-app/public/index.html my-app/public/manifest.json my-app/public/robots.txt my-app/python-backend.py my-app/src/App.css my-app/src/App.js my-app/src/App.test.js my-app/src/Chart.css my-app/src/Chart.js my-app/src/index.css my-app/src/index.js my-app/src/reportWebVitals.js my-app/src/setupTests.js README.md ``` ### Dependencies - my-app/package.json: @testing-library/dom@^10.4.0, @testing-library/jest-dom@^6.6.3, @testing-library/react@^16.3.0, @testing-library/user-event@^13.5.0, lucide-react@^0.522.0, react@^19.1.0, react-dom@^19.1.0, react-scripts@5.0.1, recharts@^2.15.4, web-vitals@^2.1.4 ### Recent commits (newest first) - Fixed a path bug to json config file - Merge branch 'main' of https://github.com/BrianStormDev/CourseCompass - Added Research Trends - hardcoded values since db is empty - chatbox fully functional now - Merge branch 'main' of https://github.com/BrianStormDev/CourseCompass - Worked on fixing prompts and the backend - idk whats going on with this repo - Removed node modules from backend - hopefully things didnt just break - Working on Plotly React Integration - added endpoint to get article content - Fixed the config for api keys (don't steal mine) - Merge branch 'main' of https://github.com/BrianStormDev/CourseCompass - Implemented a sliding window memory for chat - authors - Included option to pass custom API key - fixed the broken chat and added server url proxy to package.json - Working on backend API - Merge branch 'main' of https://github.com/BrianStormDev/CourseCompass ## Key source files (fetched from GitHub, selected and truncated for size) ### my-app/package.json ``` { "name": "my-app", "version": "0.1.0", "private": true, "dependencies": { "@testing-library/dom": "^10.4.0", "@testing-library/jest-dom": "^6.6.3", "@testing-library/react": "^16.3.0", "@testing-library/user-event": "^13.5.0", "lucide-react": "^0.522.0", "react": "^19.1.0", "react-dom": "^19.1.0", "react-scripts": "5.0.1", "recharts": "^2.15.4", "web-vitals": "^2.1.4" }, "proxy": "http://localhost:5000", "scripts": { "start": "react-scripts start", "build": "react-scripts build", "test": "react-scripts test", "eject": "react-scripts eject" }, "eslintConfig": { "extends": [ "react-app", "react-app/jest" ] }, "browserslist": { "production": [ ">0.2%", "not dead", "not op_mini all" ], "development": [ "last 1 chrome version", "last 1 firefox version", "last 1 safari version" ] } } ``` ### my-app/main.py ```python # # Option 1: FastAPI Backend (Python) # from fastapi import FastAPI, HTTPException # from fastapi.middleware.cors import CORSMiddleware # from pydantic import BaseModel # import anthropic # import os # from typing import List, Dict, Any # app = FastAPI() # # Add CORS middleware # app.add_middleware( # CORSMiddleware, # allow_origins=["http://localhost:3000"], # Your React app URL # allow_credentials=True, # allow_methods=["*"], # allow_headers=["*"], # ) # # Initialize Claude client # client = anthropic.Anthropic( # api_key=os.getenv("ANTHROPIC_API_KEY") # Set this in your environment # ) # class ChatRequest(BaseModel): # message: str # context: Dict[str, Any] # class ChatResponse(BaseModel): # response: str # @app.post("/api/claude-chat", response_model=ChatResponse) # async def chat_with_claude(request: ChatRequest): # try: # # Build context from the research data # context_str = f""" # You are a helpful AI research assistant specializing in machine learning research trends. # Current Context: # - Date: {request.context.get('date', 'N/A')} # - Recent Papers: {request.context.get('articles', [])} # - Trend Data: {request.context.get('trendData', [])} # Recent conversation: # """ # # Add conversation history # for msg in request.context.get('conversationHistory', []): # context_str += f"{msg['role'].title()}: {msg['content']}\n" # context_str += f"\nUser: {request.message}\n\nPlease provide a helpful response about ML research trends, papers, or related topics. Be concise but informative." # # Call Claude API # response = client.messages.create( # model="claude-3-sonnet-20240229", # or claude-3-opus-20240229 for more advanced responses # max_tokens=1000, # temperature=0.7, # messages=[ # { # "role": "user", # "content": context_str # } # ] # ) # return ChatResponse(response=response.content[0].text) # except Exception as e: # raise HTTPException(status_code=500, detail=f"Error calling Claude API: {str(e)}") ``` ### backend/app.py ```python import json import anthropic as Anthropic from flask import Flask, request, jsonify from flask_cors import CORS import database from langchain_community.document_loaders import ArxivLoader from ManualSlidingWindowChat import ManualSlidingWindowChat from trend_viz import * # Get the appropriate API Key with open('./config.json', 'r') as f: config = json.load(f) # Configure the app app = Flask(__name__) CORS(app) # Init the db database.init_db() # Initialize the chat api_key = config['anthropic_api_key'] chat_instance = ManualSlidingWindowChat(api_key) @app.route("/api/getArxivLinks") def getArxivLinks(): res = database.select_honorable_article_shoutouts() # return jsonify(list(map(lambda x: dict({"title": x[0], "id": x[1], "url": x[2]}), res))) return jsonify(res) @app.route("/api/claudeChat", methods=['POST']) def claudeChat(): # Get the json data from the request if request.method == "POST": # Get the data from the request data = request.get_json() # Access the data message = data.get('message') # See if we can get the appropriate chat history context = data.get('context', {}) # Unpack context date = context.get("date") articles = context.get("articles", []) initial = context.get("initial", "") # Build the prompt string prompt_parts = [] if initial: prompt_parts.append(f"Instruction: {initial}") if date: prompt_parts.append(f"Date: {date}") if articles: prompt_parts.append("Relevant Articles:") for i, article in enumerate(articles, 1): prompt_parts.append(f"{i}. {article}") if message: prompt_parts.append(f"User Message: {message}") # Final prompt to pass to Claude prompt = "\n\n".join(prompt_parts) # Send the client messages response = chat_instance.get_response(prompt) # Gets the response as a string # Process and return response response_json = jsonify({ "response": response }) return response_json @app.route("/api/get_blog_src_article/") def get_blog_src_article(article_id): loader = ArxivLoader(query=article_id, load_max_docs=1) # Replace with your arXiv ID doc = loader.load() return doc[0].page_content # @app.route("/api/chart") # def chart(): # # TODO: Put the chart generation here # return get_trend_viz(api_key) @app.route("/api/trend_data", methods=["GET"]) def trend_data(): return get_trend_viz_arr(api_key) if __name__ == '__main__': app.run(debug=True, port=5000) # Ensure port 5000 ``` ### my-app/src/index.js ```javascript import React from 'react'; import ReactDOM from 'react-dom/client'; import './index.css'; import App from './App'; import reportWebVitals from './reportWebVitals'; const root = ReactDOM.createRoot(document.getElementById('root')); root.render( ); // If you want to start measuring performance in your app, pass a function // to log results (for example: reportWebVitals(console.log)) // or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals reportWebVitals(); ``` ### my-app/src/App.js ```javascript import React, { useState, useEffect, useRef } from 'react'; import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, Legend, ResponsiveContainer } from 'recharts'; import { Send, Calendar, TrendingUp } from 'lucide-react'; import './App.css'; // import Chart from './Chart.js'; const App = () => { const [selectedDate, setSelectedDate] = useState('2024-06-21'); const [chatMessages, setChatMessages] = useState([ { role: 'assistant', content: 'Hello! Ask me anything about the recent ML research papers and trends.' } ]); const [chatInput, setChatInput] = useState(''); const [loading, setLoading] = useState(false); const [topThreeArxivTexts, setTopThreeArxivTexts] = useState([]); const chatMessagesRef = useRef(null); // Sample data - in real implementation, this would come from Python backend const [trendData, setTrendData] = useState([ { "date": "2025-03-24", "AI Security and Adversarial Defense": 11, "Causal Inference and Treatment Effects": 3, "Cross-lingual Reasoning and SQL Tasks": 28, "Emotion and Voice Recognition": 21, "Graph Representations and Network Analysis": 3, "Neuromorphic and Chemical Modeling": 8, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 3, "Reinforcement Learning and Bandits": 3, "Topic -1": 33, "Weather Prediction and Renewable Energy": 3, "Wireless Communication and Signal Processing": 1 }, { "date": "2025-03-25", "AI Security and Adversarial Defense": 13, "Causal Inference and Treatment Effects": 6, "Cross-lingual Reasoning and SQL Tasks": 56, "Emotion and Voice Recognition": 71, "Graph Representations and Network Analysis": 5, "Neuromorphic and Chemical Modeling": 23, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 1, "Reinforcement Learning and Bandits": 8, "Topic -1": 95, "Weather Prediction and Renewable Energy": 9, "Wireless Communication and Signal Processing": 2 }, { "date": "2025-03-26", "AI Security and Adversarial Defense": 13, "Causal Inference and Treatment Effects": 3, "Cross-lingual Reasoning and SQL Tasks": 36, "Emotion and Voice Recognition": 28, "Graph Representations and Network Analysis": 2, "Neuromorphic and Chemical Modeling": 7, "Non-Euclidean Clustering and Learning": 2, "Optimization and Scheduling Algorithms": 1, "Reinforcement Learning and Bandits": 3, "Topic -1": 55, "Weather Prediction and Renewable Energy": 2, "Wireless Communication and Signal Processing": 2 }, { "date": "2025-03-27", "AI Security and Adversarial Defense": 2, "Causal Inference and Treatment Effects": 2, "Cross-lingual Reasoning and SQL Tasks": 11, "Emotion and Voice Recognition": 12, "Graph Representations and Network Analysis": 3, "Neuromorphic and Chemical Modeling": 6, "Non-Euclidean Clustering and Learning": 2, "Optimization and Scheduling Algorithms": 1, "Reinforcement Learning and Bandits": 3, "Topic -1": 14, "Weather Prediction and Renewable Energy": 1, "Wireless Communication and Signal Processing": 2 }, { "date": "2025-03-28", "AI Security and Adversarial Defense": 5, "Causal Inference and Treatment Effects": 4, "Cross-lingual Reasoning and SQL Tasks": 28, "Emotion and Voice Recognition": 14, "Graph Representations and Network Analysis": 1, "Neuromorphic and Chemical Modeling": 9, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 1, "Reinforcement Learning and Bandits": 2, "Topic -1": 30, "Weather Prediction and Renewable Energy": 5, "Wireless Communication and Signal Processing": 1 }, { "date": "2025-03-29", "AI Security and Adversarial Defense": 2, "Causal Inference and Treatment Effects": 3, "Cross-lingual Reasoning and SQL Tasks": 1, "Emotion and Voice Recognition": 4, "Graph Representations and Network Analysis": 2, "Neuromorphic and Chemical Modeling": 1, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 2, "Reinforcement Learning and Bandits": 12, "Topic -1": 7, "Weather Prediction and Renewable Energy": 1, "Wireless Communication and Signal Processing": 1 }, { "date": "2025-03-30", "AI Security and Adversarial Defense": 11, "Causal Inference and Treatment Effects": 3, "Cross-lingual Reasoning and SQL Tasks": 33, "Emotion and Voice Recognition": 35, "Graph Representations and Network Analysis": 1, "Neuromorphic and Chemical Modeling": 19, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 1, "Reinforcement Learning and Bandits": 1, "Topic -1": 63, "Weather Prediction and Renewable Energy": 3, "Wireless Communication and Signal Processing": 5 }, { "date": "2025-03-31", "AI Security and Adversarial Defense": 9, "Causal Inference and Treatment Effects": 1, "Cross-lingual Reasoning and SQL Tasks": 27, "Emotion and Voice Recognition": 15, "Graph Representations and Network Analysis": 1, "Neuromorphic and Chemical Modeling": 6, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 0, "Reinforcement Learning and Bandits": 5, "Topic -1": 38, "Weather Prediction and Renewable Energy": 1, "Wireless Communication and Signal Processing": 1 }, { "date": "2025-04-01", "AI Security and Adversarial Defense": 7, "Causal Inference and Treatment Effects": 3, "Cross-lingual Reasoning and SQL Tasks": 19, "Emotion and Voice Recognition": 16, "Graph Representations and Network Analysis": 4, "Neuromorphic and Chemical Modeling": 11, "Non-Euclidean Clustering and Learning": 1, "Optimization and Scheduling Algorithms": 0, "Reinforcement Learning and Bandit [truncated — 22502 more characters] ``` ### backend/helper_classes.py ```python from dataclasses import dataclass @dataclass class Blog: blog_id: str blog_content: str score: int article_id: int @dataclass class Article: article_id: str article_title: str published_date: str url: str popularity: int ``` ### my-app/python-backend.py ```python import arxiv from datetime import datetime, timezone search = arxiv.Search( query="cat:cs.LG OR cat:cs.CL", max_results=100, sort_by=arxiv.SortCriterion.SubmittedDate, ) start_date = datetime(2024, 6, 1, tzinfo=timezone.utc) end_date = datetime(2024, 6, 21, tzinfo=timezone.utc) filtered_results = [] for result in search.results(): if start_date <= result.published <= end_date: filtered_results.append(result) print(f"Found {len(filtered_results)} papers") for paper in filtered_results: print(paper.title, paper.published) ``` ### backend/test.py ```python from flask import Flask, jsonify from flask_cors import CORS import json app = Flask(__name__) CORS(app) # Enable CORS for all routes @app.route('/api/chart', methods=['GET']) def get_chart(): """Test endpoint to verify server is working""" test_data = { "message": "Flask server is working!", "port": 5000, "endpoint": "/api/chart", "test_data": { "topics": ["AI", "Machine Learning", "Natural Language Processing"], "counts": [45, 32, 28], "timestamp": "2025-06-22" } } return jsonify(test_data) @app.route('/health', methods=['GET']) def health_check(): """Health check endpoint""" return jsonify({"status": "healthy", "port": 5000}) @app.route('/', methods=['GET']) def root(): """Root endpoint""" return jsonify({"message": "Flask backend is running on port 5000"}) if __name__ == '__main__': print("Starting Flask server on port 5000...") print("Test URLs:") print(" - http://localhost:5000/") print(" - http://localhost:5000/health") print(" - http://localhost:5000/api/chart") app.run( debug=True, port=5000, host='0.0.0.0' # Accept connections from any IP ) ``` ### backend/ManualSlidingWindowChat.py ```python import anthropic class ManualSlidingWindowChat: def __init__(self, api_key, window_size=10, max_tokens=1024): self.client = anthropic.Anthropic(api_key=api_key) self.chat_history = [] self.window_size = window_size self.model = "claude-sonnet-4-20250514" self.max_tokens = max_tokens def add_message_with_sliding(self, role, content): """Add message and manually maintain sliding window""" self.chat_history.append({ "role": role, "content": content }) # Keep only the last window_size messages if len(self.chat_history) > self.window_size: self.chat_history = self.chat_history[-self.window_size:] def get_response(self, user_message): """Get response with manual sliding window management""" # Add user message self.add_message_with_sliding("user", user_message) try: response = self.client.messages.create( model=self.model, max_tokens=self.max_tokens, messages=self.chat_history ) assistant_message = response.content[0].text # Add assistant response self.add_message_with_sliding("assistant", assistant_message) print(assistant_message) return assistant_message except anthropic.APIError as e: print(f"Anthropic API Error: {e}") return f"API Error: {str(e)}" except Exception as e: print(f"Error: {e}") return None ``` ### backend/database.py ```python import sqlite3 conn = sqlite3.connect("db.db", check_same_thread=False) from helper_classes import Blog, Article # Decorator function to skip duplicates entries when inserting into the db def unique_skip(func): def wrapper(*args, **kwargs): try: return func(*args, **kwargs) except sqlite3.IntegrityError as e: if "UNIQUE constraint failed" not in str(e): raise e return wrapper def init_db(): """Creates all the tables in the db""" cur = conn.cursor() # Each blog corresponds to a single article # Pick the paper with the highest `score` to write a blog post about # The score is based on num of citations, how goated is the author, recency, and maybe similarity to the most popular topic found in the trend analysis cur.execute(""" CREATE TABLE IF NOT EXISTS articles( article_id TEXT PRIMARY KEY UNIQUE, article_title TEXT, published_date TEXT, url TEXT, popularity INTEGER ) """) cur.execute(""" CREATE TABLE IF NOT EXISTS blogs( blog_id INTEGER PRIMARY KEY UNIQUE, blog_content TEXT, score INTEGER, article_id INTEGER REFERENCES articles(article_id) ON DELETE CASCADE ) """) # Table for the trends # Feed to to BERTopic, which is a semantic analysis tool # We want to plot the popularity of the trends cur.close() # The 5 articles that we display on the sidebar are based on popularity def select_honorable_article_shoutouts(): # cur = conn.cursor() # cur.execute("SELECT article_title, article_id, url FROM articles ORDER BY popularity DESC LIMIT 5"); # res = cur.fetchall() # cur.close() # return res return [ { "title": 'Attention Is All You Need: A Comprehensive Survey', "id": '2406.12345', "url": 'https://arxiv.org/abs/2406.12345' }, { "title": 'Scaling Laws for Neural Language Models in 2024', "id": '2406.12346', "url": 'https://arxiv.org/abs/2406.12346' }, { "title": 'Diffusion Models: Theory and Applications', "id": '2406.12347', "url": 'https://arxiv.org/abs/2406.12347' }, { "title": 'Reinforcement Learning with Human Feedback', "id": '2406.12348', "url": 'https://arxiv.org/abs/2406.12348' }, { "title": 'Vision Transformers: The Next Generation', "id": '2406.12349', "url": 'https://arxiv.org/abs/2406.12349' } ] @unique_skip def insert_new_article(a: Article): cur = conn.cursor() cur.execute("INSERT INTO articles(article_id, article_title, published_date, url, popularity) VALUES(?, ?, ?, ?, ?)", (a.article_id, a.article_title, a.published_date, a.url, a.popularity)) conn.commit() cur.close() @unique_skip def insert_new_blog(b: Blog): cur = conn.cursor() cur.execute("INSERT INTO blogs(blog_id, blog_content, score, article_id) VALUES(?, ?, ?, ?)", (b.blog_id, b.blog_content, b.score, b.article_id)) conn.commit() cur.close() ``` [9 more indexed source files omitted to keep this export small. The full file list is in the Codebase structure section above.]