Project Info
This project did not submit a demo video on Devpost.
Inspiration
Social media algorithms shape what we see, often reinforcing the habits we’re trying to break. Just like your environment and the people around you influence your growth, your feed does too. ReFocus was inspired by the idea that recovery and self-improvement start with what surrounds you online. We wanted to build a way to take control back from the algorithm and help users create a cleaner, healthier digital space.
What it does
ReFocus helps users retrain their social media feeds by identifying and skipping suggestive or distracting content. It works by detecting "trigger" categories—content types the user wants to avoid—and automatically acting on them. For example, if a user is trying to avoid suggestive videos, ReFocus helps skip and filter out those posts, gradually teaching the platform’s algorithm to show less of that content over time.
How we built it
We built ReFocus using LiveKit for real-time screen sharing and interaction. When a user shares their screen, the app connects to a Node.js server that creates a new WebRTC room. From there, we used RoomIO for handling communication between the agent and user participants through audio and video tracks. We enabled live video input via RoomInputOptions(video_enabled=True), allowing the agent to receive frames from the user’s screen and classify them at regular intervals (1 FPS while speaking, 1 frame every 3 seconds otherwise). These frames are resized to 1024×1024 and encoded as JPEG for model processing. The backend uses FastAPI (Python) to facilitate interactions between the AI agent and the client, handling classification requests and trigger detection.
Challenges we ran into
Getting LiveKit to run smoothly under unstable network conditions (pro tip: don’t test with bad Wi-Fi). Getting LiveKit to run smoothly under unstable network conditions (pro tip: don’t test with bad Wi-Fi). Integrating WebRTC and ensuring real-time responsiveness across browser environments. Integrating WebRTC and ensuring real-time responsiveness across browser environments. Managing the workflow logic — deciding when and where the agent should click, skip, or observe. Managing the workflow logic — deciding when and where the agent should click, skip, or observe.
Accomplishments we're proud of
Successfully got LiveKit working end-to-end with real-time screen streaming. Successfully got LiveKit working end-to-end with real-time screen streaming. Built a functioning pipeline for detecting visual triggers and responding automatically. Built a functioning pipeline for detecting visual triggers and responding automatically. Established a foundation for behavior-driven feed retraining. Established a foundation for behavior-driven feed retraining. Demonstrated that feed detoxification can be automated in a way that supports recovery and focus. Demonstrated that feed detoxification can be automated in a way that supports recovery and focus.
What we learned
We learned how powerful agent-driven automation can be when combined with real-time video and audio analysis. We also realized how critical workflow design is for making automation safe, ethical, and responsive. Most importantly, we gained insight into the potential of using technology not just for engagement—but for digital recovery and mindfulness.
What's next
Expand our content classification model to detect a broader range of triggers. Expand our content classification model to detect a broader range of triggers. Refine agent interactions to improve accuracy and reliability across platforms. Refine agent interactions to improve accuracy and reliability across platforms. Add user customization features for defining personal triggers and recovery goals. Add user customization features for defining personal triggers and recovery goals. Eventually, create a browser extension that passively retrains your feed as you scroll—helping you build a healthier digital environment, one skip at a time. Eventually, create a browser extension that passively retrains your feed as you scroll—helping you build a healthier digital environment, one skip at a time.
Content Filter - Take Control of Your Feed
Cal Hacks 2024 Project
A Chrome extension powered by AI that gives users control over social media content algorithms. Filter out unwanted content using natural language descriptions.
Overview
Social media algorithms are like black boxes - users have little control over what they see. This project empowers users to take back control by specifying content they don't want to see in natural language. The system uses AI vision models to analyze video content in real-time and automatically clicks "Not interested" when triggers are detected.
Features
- Natural Language Filters: Describe what you don't want to see (e.g., "smoking", "violence")
- Real-time Video Analysis: AI analyzes video frames as you browse
- Multi-platform Support: Works on YouTube, Instagram, TikTok, Facebook, Twitter, and more
- Privacy-focused: Video analysis happens in real-time, no data stored
- Intelligent Button Detection: Automatically finds and clicks "Not interested" buttons
- Visual Feedback: Track how many items have been filtered
Technology Stack
Chrome Extension
- JavaScript/TypeScript
- Chrome Extension API (Manifest V3)
- WebRTC for screen capture
- LiveKit Client SDK
LiveKit Agent (Python)
- LiveKit Agents SDK
- Anthropic Claude 3.5 Sonnet (vision model)
- PIL/Pillow for image processing
- Bright Data MCP for DOM intelligence
Communication
- LiveKit Cloud for real-time WebRTC
- Data channels for bidirectional messaging
Architecture
┌─────────────────────────────────────┐
│ Chrome Extension │
│ ┌──────────────────────────────┐ │
│ │ Popup UI (Trigger Input) │ │
│ └──────────────────────────────┘ │
│ ┌──────────────────────────────┐ │
│ │ Content Script │ │
│ │ (DOM Interaction) │ │
│ └──────────────────────────────┘ │
│ ┌──────────────────────────────┐ │
│ │ Background Service Worker │ │
│ │ (Screen Capture + WebRTC) │ │
│ └──────────────────────────────┘ │
└────────────┬────────────────────────┘
│ WebRTC Video Stream
▼
┌─────────────────────────────────────┐
│ LiveKit Cloud │
│ ┌──────────────────────────────┐ │
│ │ LiveKit Agent (Python) │ │
│ │ - Video Analysis │ │
│ │ - Trigger Detection │ │
│ │ - Command Generation │ │
│ └────────┬─────────────────────┘ │
└───────────┼─────────────────────────┘
│
▼
┌─────────────────────────────────────┐
│ Bright Data MCP Server │
│ - Webpage Context │
│ - DOM Element Location │
└─────────────────────────────────────┘
Quick Start
Prerequisites
- LiveKit Cloud Account - https://livekit.io
- Anthropic Claude API Key - https://console.anthropic.com
- Node.js 18+ and Python 3.10+
- UV (Python package manager) - https://astral.sh/uv
Setup
-
Clone the repository
cd Calhacks -
Configure environment
cp .env.example .env # Edit .env with your API keys -
Set up the agent
cd agent uv pip install -r requirements.txt python main.py -
Build the extension
cd extension npm install npm run build -
Load extension in Chrome
- Go to
chrome://extensions/ - Enable "Developer mode"
- Click "Load unpacked"
- Select
extension/distfolder
- Go to
-
Start filtering!
- Click the extension icon
- Add your content filters
- Click "Start Monitoring"
- Browse social media
Documentation
- Setup Guide - Detailed setup instructions
- API Documentation - API reference and message protocols
- Implementation Plan - Complete implementation details
Project Structure
Calhacks/
├── extension/ # Chrome Extension
│ ├── popup/ # UI for managing filters
│ ├── content/ # Content scripts for DOM
│ ├── background/ # Service worker + WebRTC
│ └── lib/ # Shared utilities
│
├── agent/ # LiveKit Agent (Python)
│ ├── main.py # Agent entry point
│ ├── video_analyzer.py # Vision LLM integration
│ ├── mcp_client.py # Bright Data MCP
│ └── command_sender.py # Extension communication
│
├── docs/ # Documentation
├── mcp-config/ # MCP server config
└── plan.md # Detailed implementation plan
How It Works
-
User inputs filters: User describes unwanted content in natural language (e.g., "smoking", "violence")
-
Extension captures screen: When monitoring starts, the extension captures the browser tab's video stream
-
Stream to LiveKit: Video is sent via WebRTC to LiveKit Cloud where the agent receives it
-
AI analyzes frames: The agent processes video frames using GPT-4V or Claude vision models
-
Trigger detection: When unwanted content is detected, the agent identifies it
-
Find action button: Agent uses Bright Data MCP to find "Not interested" buttons on the page
-
Execute action: Agent sends click command back to extension, which executes the click
-
Continue monitoring: Process continues in real-time as user browses
Supported Platforms
- ✅ YouTube (videos and shorts)
- ✅ Instagram (feed and reels)
- ✅ TikTok (For You page)
- ✅ Facebook (feed)
- ✅ Twitter/X (timeline)
- ✅ Reddit (feed)
- ✅ Generic support for other platforms
Configuration
See .env.example for all configuration options:
# Required
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_key
LIVEKIT_API_SECRET=your_secret
ANTHROPIC_API_KEY=sk-ant-...
# Optional
FRAME_SKIP_COUNT=2 # Process every 3rd frame
IMAGE_MAX_SIZE=1024 # Max image dimension
MIN_CONFIDENCE_THRESHOLD=0.7 # Detection threshold
Cost Estimates
- Anthropic Claude 3.5 Sonnet: ~$0.015 per 1000 frames
- LiveKit Cloud: Free tier available, then $0.01/min
Tips to reduce costs:
- Increase
FRAME_SKIP_COUNT - Reduce
IMAGE_MAX_SIZE - Use Claude 3 Haiku for a cheaper, faster option
Development
Building Extension
cd extension
npm run dev # Watch mode for development
npm run build # Production build
Running Agent
cd agent
python main.py
# Or with debug logging
LOG_LEVEL=DEBUG python main.py
Debugging
- Extension logs: Chrome DevTools Console (F12)
- Agent logs: Terminal output
- LiveKit dashboard: https://cloud.livekit.io
Known Limitations
- Requires screen capture permission
- Vision API costs for high usage
- Detection accuracy depends on model quality
- May not work on all websites due to CSP policies
- Requires active internet connection
Future Enhancements
- Local ML inference (reduce API costs)
- Mobile browser support
- Collaborative filter lists
- Advanced rules (time-based, contextual)
- Performance optimizations
- Multi-language support
Contributing
This is a Cal Hacks 2024 hackathon project. Contributions and suggestions are welcome!
Privacy & Security
- Video frames are processed in real-time
- No video data is stored or persisted
- User filters are stored locally in Chrome storage
- All communications use encrypted WebRTC/WSS
- API keys should be kept secure and rotated regularly
License
This project was created for Cal Hacks 2024.
Team
Built with ❤️ for Cal Hacks 2024
Acknowledgments
- LiveKit - Real-time communication infrastructure
- OpenAI/Anthropic - Vision AI models
- Bright Data - Web intelligence via MCP
- Cal Hacks - For hosting an amazing hackathon!
Support
For issues or questions:
- Check docs/SETUP.md for troubleshooting
- Review docs/API.md for technical details
- Check plan.md for implementation details
Note: This project uses AI vision models which may have rate limits and costs. Please review the pricing for your chosen provider and configure frame processing settings accordingly.
Analysis
View
Metric
- 1
- 1
Figures cover GitHub contributors during the hackathon window. A co-authored commit counts in full for each author, so per-member totals add up to more than the whole-team figures.
Technology
- CSSIn code
- HTMLIn code
- JavaScriptIn code
- OpenAIIn code
- PythonIn code
- FastAPIClaimed
- Node.jsClaimed
5 of 7 appear in the indexed code. 2 claimed on Devpost could not be matched to code, which may simply mean the tool leaves no trace in the repository.
AI coding agents
No AI coding agent signals were found in this repository.
Detected from committed agent config files and commit authorship. Absence of a signal is not proof an agent was unused.
Codebase size
Source size
142 KB
Source files
22
Counts recognized source files only; vendored directories, binaries and lockfiles are excluded, so this is smaller than the repository on disk.
Repository
Steven-Hsu1/CalHacks
31 files · 621 KB · @ 9c6cf48
Structure
Interface
3 files · 10%Screens, components and styles rendered to the user.
Application logic
16 files · 52%Domain rules, services and shared utilities.
Supporting
Layers are inferred from where files sit in the tree, not from reading the code. A project that names its directories unconventionally will read oddly here — open the file browser to check anything the diagram implies.
Languages
- JavaScript43%
- Python42%
- Markdown11%
- CSS3%
- HTML2%
Share of indexed source by file size. Binary and vendored files are excluded.
Dependencies
agent/pyproject.toml
pypi · 9- aiohttp
- livekit
- livekit-agents[anthropic]
- openai
- pillow
- python-dotenv
- +3 more
agent/requirements.txt
pypi · 7- aiohttp
- livekit
- livekit-agents
- openai
- pillow
- python-dotenv
- uvloop
extension/package.json
npm · 6- dotenv
- livekit-client
- livekit-server-sdk
- +3 more
Declared in the repository’s manifests at the indexed commit. A declared package is not proof it is used, and runtime dependencies are listed first.
This project’s features have not been analysed yet.
Export this project's context (description, README, evidence, key source files) to chat with an AI agent elsewhere.